First boot doesn't recover if system restarts while snapd is starting

Hi,

Recently, I am testing if the system can recover from an uncompleted first-boot.
Then I found if the interrupt is at [ OK ] Started /tmp/tmp.q8QtKH3Vr0/usr/lib/snapd/snapd., then the system will not recover. If the interrupt happens later, then system can recover correctly.

I use qemu and amd64 UC18 image to test. The procedures are listed as follow:

  1. start qemu
  2. wait till it show “[ OK ] Started /tmp/tmp.q8QtKH3Vr0/usr/lib/snapd/snapd.” and close qemu.
  3. restart qemu and it will show “[FAILED] Failed to start Wait until snapd is fully seeded (core18).”

Part of first boot log

[ OK ] Listening on /tmp/tmp.q8QtKH3Vr0/usr/lib/snapd/snapd.
         Starting /tmp/tmp.q8QtKH3Vr0/usr/lib/snapd/snapd...
[ OK ] Started WPA supplicant.
[ OK ] Reached target Network.
         Starting OpenBSD Secure Shell server...
         Starting Permit User Sessions...
[ OK ] Started Permit User Sessions.
[ OK ] Started Serial Getty on ttyS0.
[ OK ] Reached target Login Prompts.
[ OK ] Started OpenBSD Secure Shell server.
[ OK ] Started /tmp/tmp.q8QtKH3Vr0/usr/lib/snapd/snapd.

Part of restart first-boot log

[ OK ] Started Start the snapd services from the snapd snap.
         Starting Wait until snapd is fully seeded (core18)...
[FAILED] Failed to start Wait until snapd is fully seeded (core18).
See 'systemctl status snapd.seeded.service' for details.

core18.start-snapd.service will execute /usr/lib/core18/run-snapd-from-snap which will check if /var/lib/snapd/state.json exists.
When this issue happens, state.json is just initialized and snaps are not ready yet. I think it’s why the snapd.seeded.service failed (because /usr/bin/snap is not ready as well).

state.json

{"data":{"patch-level":6,"patch-sublevel":2,"refresh-privacy-key":"BMGDB7Srtp0NxKV2"},"changes":{},"tasks":{},"last-change-id":0,"last-task-id":0,"last-lane-id":0}

Is this expected or should be fixed?

This is unexpected and we actually bumped into this very issue yesterday while debugging something else. We have a good feeling of what needs to be done to fix this. W will report a bug for tracking and update this thread.

1 Like

This is now bug https://bugs.launchpad.net/snapd/+bug/1845310