Are validation sets not supposed to block install of a different snap revision?

snapd
1

We have a custom device model that is pointed to our private store and we want to pin the revision of third party snaps (like core24, snapd, network-manager, etc) so that we can control on our own cadence to make sure they don’t bring in any breaking changes. However, in our own testing, we were able to manually install a different revision of a snap (this was on a offline device). From the manage validation set docs in “Enforcing assertion validity”, it says:

When validation set validity is enforced, snapd will block any operations that would result in snap revisions violating the validation set’s constraints and rendering it invalid.

which seems counter to what we saw. Is the doc wrong or is this a bug? It seems like the revision should be enforced no matter what. Note that offline is important to us because we have different kinds of customers - some are OK with online devices and some are not. We’ve built out a device agent to support offline updates and would like to have consistent behaviour when it comes to validation sets.

Here is what I did to test. I created a test validation set:

account-id: <separate-account-for-validations>
name: spike-thirdparty
sequence: 1
# The revision for this validation set
# revision: 0
snaps:
- id: dwTAh7MZZ01zyriOZErqd1JynQLiOGvM
  name: core24
  presence: required
  revision: 1588
- id: dVK2PZeOLKA7vf1WPCap9F8luxTk9Oll
  name: avahi
  presence: required
  revision: 581

Our model assertion points to that validation set:

  "validation-sets": [
    {
      "account-id": "<separate-account-for-validations>",
      "name": "spike-thirdparty",
      "mode": "enforce"
    }

Create image from the signed model and boot the image (device is not Internet connected). The expected revisions were there. I copied over an assertion and snap file for a newer version of avahi (revision 678), snap ack the assertion file and then snap install the snap file. At this point, I would expect this to fail but it doesn’t. When the device is online and try to refresh to a different revision:

snap refresh --revision=678 avahi

I do get an error:

error: cannot refresh "avahi": cannot update snap "avahi" to revision 678
       without --ignore-validation, revision 581 is required by validation sets: ...

I would have expected this error to also happen with snap install but it doesn’t. Let me know if you need any more information.

2

Some more information here as I did some more experimentation especially around how to do an offline update of snaps and the validation set. You need to:

  • Ack assertion and install snaps that have new revisions first
    • At this point, the validation set is marked as invalid
  • Ack new sequence of validation set
  • snap validate –enforce <validation_set>
    • At this point, the validation set is marked as valid

So it seems like you need to be able to install a snap that will make the validation set invalid to support this use case so that could explain why an install is not blocked? If do the above in a different order (ack/snap validation the validation set first), it shows this error:

error: cannot apply validation set: cannot enforce validation set: validation sets assertions are not met:

  • snaps at wrong revisions:
    • core24 (required at revision 1644 by sets /<validation_set_name>)