I (intentionally) started my image with a gadget snap whose prepare-device hook points at a non-existent serial vault.
Not surprisingly. ‘snap changes’ shows the “Initialize device” change seemingly stuck in “Doing”.
I am testing whether I can install an upated gadget with a prepare-device hook that is fixed and now points at a legitmate serial vault so the system can receive a proper serial assertion.
So, I installed the updated gadget (in terminal with --dangerous), (and even rebooted), but the change (above) is still stuck in “Doing” and the change detail shows that it is still using (periodically) the URL of the original gadget prepare-device hook, not the newly installed correct one.
I’d very much like to be able to recover some deployed systems that have a boilerplate (incorrect) prepare-device hook to point them at a proper serial vault.
As we discussed in the call and @pedronis figured out, the issue here is that the device is attempting to talk to a broken serial vault and failing, and then it retries every 60 seconds so that it can proceed with the normal device initialization. It will continue doing that even if the gadget is updated, but unfortunately the fact the hook changed will not immediately be observed by the system because the hook already run, and it finished successfully. That said, at the moment any tasks that are being retried for more than a week will be automatically aborted by the system, and after that snapd will figure that the device was not yet initialized, and that there’s no tasks in progress attempting to fix that, and it will respawn the initialization routines again, running the hook once more and obtaining the correct serial vault URL.
I confirm that after installing an updated gadget snap whose prepare-device hook points at a proper serial vault & using ‘snap abort CHANGE’ (to kill the “Initialize Device” task), the device tried again, this time succeeding in obtaining the serial assertion.
