It would be really useful to be able to specify a particular sequence of a validation-set at image build time, but without using the sequence number of the validation set in the model assertion itself. If you specify the sequence number in the model assertion the device is locked to that sequence forever, which isn’t desirable for my purposes.
Right now, ubuntu-image just get’s the most recent sequence if no sequence is specified. I’m wondering if a patch that allows specifying a validation-set name and number [ as can be done for snaps currently ] is all that’s required to support this, or if there is something fundamental I’m missing here that would make that feature impractical.
Or, perhaps I’m missing some other way to do this that already exists ?
We use validation-sets to control revisions of third party snaps used across devices [ snaps we don’t release ourselves, which includes Canonical snaps ], especially in production. We have an agent on devices that sets which sequence number to use for a specific validation-set that is included in the model. We update the config which that agent uses via snapcraft progressive release mechanism. This allows us to make changes across devices in a controlled way across the fleet, monitoring how the release is going, and increasing the percentage if all is well for a given snap revision.
This process works well and we are happy with it. Because we use that process, we don’t want to specify the sequence number for a given validation-set in the model assertion. because as the documentation you linked states:
If a sequence is specified, then only that sequence is permitted for the lifetime of the device
That’s fine, we don’t specify a sequence number, but that means the latest one is always used when building images from the model, which is not always desirable for a number or reasons which I will list some of below.
It’s also worth making clear that we tend to build new images from current models assertions regularly. We do that because we find that older images can have issues during bootstrap, partly because of the continuous delivery we do which is split across multiple different components [ older images force a kind of “big bang” upgrade that wasn’t engineered for ]. We build devices in the field regulatory, either because they are new, or as part of a support process. Having images that more closely matches the running state makes that process smoother.
The reasons why blindly including the latest sequence number for a given validation-set in a model can be a problem:
We may not have actually started the progressive release that deploys that sequence across production yet.
We deployed the latest sequence, but rolled it back because of an issue [ we can mitigate that instead by rolling forward to a new validation-set sequence with the older revisions, but that’s not always ideal ].
Validation-sets don’t really handle architecture, just revisions. If you are controlling revisions for a multi arch snap, you can either create multiple architecture specific validation-sets, or handle them all in the same validation set but with a different known sequence for each architecture. There’s pros/cons to either approach, but If we chose the second one then the latest sequence number may not have the correct architecture, which would of course break the image build process.
Hope that helps explain a bit.
This is the PR. It’s been reviewed, but I’m still waiting for a final sign off and to get it merged. Maybe you can help with that ?
