Validation sets

A validation set is an assertion that lists specific snaps that are either required to be installed together or are permitted to be installed together on a device or system.

One or more validation sets can be used to ensure only specific snaps are installed, and optionally, only specific snaps at fixed revisions. They can help a set of interdependent snaps maintain their testing and certification integrity, as well as help orchestrate their updates. But they can equally be used to simplify dependency deployment and to help manage devices.

Prerequisites

Validation set functionality is currently under active development and there are several considerations that need to be made before using it:

  • A developer account is also required, along with your developer id.
    (see Create a developer account for further details)
  • Snapd version 2.50 and Snapcraft version 4.7, or newer, are required.
  • Currently, in order to enforce a validation set, the following is also required:
    • a brand store account
    • the validation setā€™s name, account-id and listed snaps, need to be added to an allowlist in the store backends (this is done by filing a brand store support ticket).
    • Snapd version 2.54

See below for further details on the following:


Creating a validation set

For devices running Ubuntu Core, a validation set can be declared as part of the model definition.

To create a validation set from the command line, use the snapcraft edit-validation-sets command:

snapcraft edit-validation-sets <account-id> <set-name> <sequence>

This command requires your developer account id, an arbitrary name for the validation set, and a sequence number. The sequence number starts at 1 and is incremented at the developerā€™s discretion when thereā€™s a substantial change or update to the validation set:

snapcraft edit-validation-sets xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f myset1 1

An additional --key-name argument can be used to specify a key other than the default.

A text editor will open containing a template for a validation set assertion that needs to be filled in by the developer issuing the assertion.

validation-set template
account-id: <account-id>
name: <set-name>
sequence: <sequence>
# The revision for this validation set
# revision: 0
snaps:
#  - name: <name>  # The name of the snap.
#    id:   <id>    # The ID of the snap. Optional, defaults to the current ID for
                   # the provided name.
#    presence: [required|optional|invalid]  # Optional, defaults to required.
#    revision: <n> # The revision of the snap. Optional.

The template validation set assertion needs to be populated with the details of the snaps you wish to include in the set. These are listed beneath the snaps: section, and each snap can use the following fields:

  • name (required): The name of the snap, as you find on the store or in snap search.
  • id (optional): The unique snap-id of the snap (see snap info <snap name> ). Defaults to the snap-id of the named snap.
  • presence (optional): Can be either required, optional or invalid. required snaps need to be installed, optional snaps are permitted to be installed and invalid snaps explicitly must not be installed. Defaults to required.
  • revision (optional): Specifies which revision of the snap needs to be installed.

The following is a populated example of a validation set assertion:

account-id: xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f
name: myset1
# revision: 0
sequence: 1
snaps:
  - name: hello-world
    id: buPKUD3TKqCOgLEjjHx5kSiCpIs5cMuQ
    presence: required
  - name: test-snapd-base-bare
    id: oXC9AkhtCxhlY80KZA3peZzWbnO4xPOT
    presence: optional
  - name: bare
    id: EISPgh06mRh1vordZY9OZ34QHdd7OrdR
    presence: optional

We recommend making a copy of the saved validation set assertion before closing the editor. Closing the editor will first check the integrity of the assertion before automatically uploading it to the store.

To modify the assertion at a later point, run the same snapcraft edit-validation-sets command with the same name but an incremented sequence number and/or revision.

Listing validation sets

Use the snapcraft list-validation-sets command to check which validation sets area available in the store:

$ snapcraft list-validation-sets
Account-ID                       Name      Sequence  Revision  When
xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f myset1    1         0         2021-04-08
xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f testset1  2         0         2021-03-31

To list only validation-sets with a specific set name, use the additional --name argument:

$ snapcraft list-validation-sets --name myset1
Account-ID                       Name      Sequence  Revision  When
xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f myset1    1         0         2021-04-08

An additional --sequence argument can be used to list validation sets with a specific sequence number:

$ snapcraft list-validation-sets --name myset1 --sequence 1
Account-ID                       Name      Sequence  Revision  When
xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f myset1    1         0         2021-04-08

By default, only the most latest validation sets are listed. To list every validation set available, add the --all argument.

Monitoring assertion validity

The snap validate --monitor command is used to enable monitoring of a validation assertion on the system; in this mode the constraints of the assertion are not enforced (e.g. snaps may get automatically refreshed to newer revisions that make the assertion invalid as show in the next example):

snap validate --monitor xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f/testset1

The snap validate command, with no further arguments, checks whether the snaps: rules for all validation set assertions on the store are valid for the system:

$ snap validate
Validation                                 Mode     Seq  Current    Notes
xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f/myset1    monitor  1    valid  
xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f/testset1  monitor  2    invalid

An assertion is invalid if snaps in the system do not satisfy the constraints of the assertion, such as if required snaps are missing or whether unwanted snaps are present. Multiple validation sets can be used, as shown above, as long as they donā€™t have conflicting constraints and that they can cover different sets of snaps.

A specific validation set can be checked with snap validate <account id>/<validation set name>, with an optional sequence point set by adding =<sequence> to the validation set name:

$ snap validate xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f/myset1=1
valid

A validation set assertion can be pinned by the system administrator at the given sequence number:

snap validate --monitor xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f/testset1=3

A pinned validation set is kept at the given sequence number, even if thereā€™s a higher sequence number in the store. However, the validation will be updated to a newer version if one becomes available with the same sequence number.

Monitor mode validation requires a manual action (snap validate, as shown above), but nothing is enforced in the system. Only when enforce mode has been implemented will validation sets have an impact on the system and will prevent installing/removing snaps that violate an assertionā€™s constraints.

Finally, to remove a validation set from the system, use the --forget argument:

snap validate --forget xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f/myset1

Enforcing assertion validity

When enforcing a validation set, snapd will ensure that:

  • Snaps required by a validation set are both present and, if specified, at the correct revision. Attempting to remove a required snap will result in an error and the process will be rejected.
  • Snaps are only refreshed to newer revisions if they continue to satisfy whatever validation sets are in use.
  • Invalid snaps are not allowed to be installed. Attempting to install them will result in an error.

A validation set can be enforced by adding the --enforce argument to the ā€˜snap validateā€™ command:

snap validate --enforce xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f/myset1

Every snap required by a validation set needs to be installed before enforcing is enabled. The snap daemon will neither install missing snaps nor remove invalid snaps. If there are snaps missing, or invalid snaps installed, the assertion will simply become invalid. Itā€™s possible to use the --refresh --enforce arguments to install or refresh any snaps required for the assertion to be valid. Note that, if the assertion requires snaps to be removed, the --refresh --enforce request will not remove them and will instead quit without making any changes.

snap validate --refresh --enforce xSfWKGdLoQBoQx88vIM1MpbFNMq53t1f/myset1

After enforcement is enabled, snapd ensures the consistency of the enforced validation sets, and the snaps they reference, during install, refresh and remove operations.

During auto-refreshes, or manual refreshes, enforced validation set assertions on the system may be refreshed to newer revisions if the assertion is:

  • present in the store
  • not pinned to a specific sequence

An assertion will move to the latest sequence if present in the store and if the installed snaps, including any newer revisions in the store, still satisfy their respective validation set assertions.

In this way, the snapcraft edit-validation-sets command can be used to control the updates of multiple snaps at the same time.

For brief periods during multi-snap updates, different snap revisions, from previous and incoming validation set sequence points, can co-exist. validation-sets enforcement is not intended to deal with any breaking hard version dependencies during transitions.

As with monitor mode, enforcing can be disabled for select validation sets with the ā€˜snap validate --forgetā€™ command.

When using snap install and snap refresh, the --ignore-validation flag can be added to bypass validation set enforcement for the snaps affected. Doing so will ignore the validation of the given snap, and for subsequent refresh operations. This may result in the validation set becoming invalid in snap validate output.

Not really a docs issue, but given the valid command ā€œsnap validate ā€¦ā€ described on this page, it is curious that ā€œsnap --helpā€ makes no mention of the ā€œvalidateā€ subcommand. Even ā€œsnap help --allā€ says nothing about it.

Also (and Iā€™ve mentioned this earlier), other than a link to this page from ā€œWhatā€™s Newā€, there is no ToC entry that would lead a reader here.

Hi @degville,

i created a validation set using my developer accout on my pc. Now if i want to enforce it on a Ubuntu Core system, how do I do that? There is no enforce validation on snapd rest api. so how do i communicate with the UCore system to enforce validation. Moreover when i tried to enfore it over there manually, it returns

error: cannot apply validation set: invalid mode ā€œenforceā€

What version of snapd do you have (snap --version)?

snap    2.53.4
snapd   2.53.4
series  16
kernel  5.4.0-1050-raspi

Right. 2.53.x only supports --monitor. Enforcing became available with 2.54 (I just checked in the code) and this documentation was updated, but we forgot to adjust snapd version number; sorry for the confusion.

1 Like

I updated snapd and then it worked. also i have to enforce the validation manually on the UCore system. Is there a way to do this though snapd rest api?

Yes there is (snap command talks via REST API for everything it does). The documentation for this API should become available very soon at https://snapcraft.io/docs/snapd-api

1 Like

The documentation for this has now been added to our REST API reference: validation-sets

2 Likes

As I test it, ā€œsnap validate --forgetā€ does not remove a vset from the system, it just stops enforcing it, correct? It still shows up in the list of available vsets so that it can be set to enforcing again. Is that correct?

Oh, wait, I see that further down, thatā€™s explained. Thatā€™s a really unfortunate choice of option, --forget; is it too late to change it to something like ā€œā€“unenforceā€?

Hi, I was testing how to use validation sets.

I first install all the snaps at the revision i needed and I was able to enforce the validation set.

After that i updated the validation set to have a snap of a different revision. Now when i do : snap refresh and snap validate all i get is that the validation set is invalid now and enforced.

How can i get it to refresh automatically to required revision, shouldnt it do that?

@rahul-tt Are you using brand store?

yes, i am using a brand store

Iā€™ve observed something similar. Below, dev machine is Ubuntu Desktop 20.04, and target machine is UC20 pointing to a private brand store.

  1. On dev machine: Publish snaps acme-foo and acme-bar in private brand store. Upload & release revision 1 of acme-foo and revision 1 of acme-bar to the respective latest/stable channels.

  2. On dev machine: Create Ubuntu Core image, which includes snaps acme-foo following latest/stable and acme-bar following latest/stable in model assertion.

  3. On target machine: Boot image. Allow system to prepare device & perform initial refresh (no updates expected to acme-foo & acme-bar). Place a refresh hold for one week (arbitrary time, simply to prevent unexpected auto-refreshes during test).

  4. On dev machine: Run snapcraft edit-validation-sets acme vs-baz 1, to create:

account-id: acme
name: vs-baz
sequence: 1
# The revision for this validation set
# revision: 0
snaps:
  - name: acme-foo
    id:   f00a1phanum3r1cstr1ng
    presence: required
    revision: 1
  - name: acme-bar
    id:  bara1phanum3r1cstr1ng
    presence: required
    revision: 1
  1. On target machine: Run snap validate --enforce acme/vs-baz=1. This succeeds (as the set is simply describing the current state of the latest/stable channels, which the device is following and up to date on.) snap validate confirms that the set is valid and being enforced.

  2. On dev machine: Upload and release revision 2 of acme-bar to store. Release to latest/stable.

  3. On target machine: Run snap refresh, and observe that no new snap revisions are installed, as expected (because while revision 2 is available on latest/stable the currently enforced validation set prevents it from being installed)

  4. On dev machine: Run snapcraft edit-validation-sets acme vs-baz 1, to create

account-id: acme
name: vs-baz
sequence: 1
# The revision for this validation set
# revision: 1
snaps:
  - name: acme-foo
    id:   f00a1phanum3r1cstr1ng
    presence: required
    revision: 1
  - name: acme-bar
    id:  bara1phanum3r1cstr1ng
    presence: required
    revision: 2
  1. On target machine: run snap refresh.

    EXPECTED BEHAVIOR: acme-bar refreshes to revision 2, as the latest revision of acme/vs-baz=1 allows. acme/vs-baz=1 remains valid & enforced.

    OBSERVED BEHAVIOR: acme/vs-baz=1 becomes invalid. acme-bar is also not refreshed to revision 2. snap known validation-set indicates the device is aware of latest revision of vs-baz but something breaks along the way

This is exactly what i am facing. Thanks for explaining with an example

@alexclewontin Thanks for providing detailed steps and inputs. Could you please attach the list of snap changes and the list of tasks (snap change <ID>) for the last snap refresh; also, journalctl -u snapd for that period may be helpful if you still have it?

@rahul-tt and @alexclewontin There is an additional requirement currently in place (besides brand store) for enforcing of validation sets by snap store that was missing in the original requirements, the first post in this thread has just been updated to reflect that - see the points after " Currently, in order to enforce a validation set, the following is also requiredā€¦". Sorry for the confusion.