Core on an intel NUC (D54250WYK) installation, update for pc-kernel are broken.
The snap will update, require a reboot, but the reboot won’t happen; I must use the hardware reboot button which I suppose trigger a rollback on pc-kernel.
I’m not “refreshing” manually anymore, but snapd will do it itself anytime soon, means my home server can be softbricked at anytime (well, until it is rebooted)
cm-t@localhost:~$ sudo snap refresh
2024-06-29T15:55:19Z INFO Task set to wait until a system restart allows to
continue
Make snap "pc-kernel" (1892) available to the system \
Broadcast message from root@localhost (Sat 2024-06-29 15:55:19 UTC):
reboot scheduled to update the system
The system is going down for reboot at Sat 2024-06-29 16:05:19 UTC!
2024-06-29T15:55:19Z INFO Task set to wait until a system restart allows to
continue
Change 341 waiting on external action to be completed
cm-t@localhost:~$
Broadcast message from root@localhost (Sat 2024-06-29 15:55:22 UTC):
reboot scheduled to update the system
The system is going down for reboot at Sat 2024-06-29 15:56:22 UTC!
snap list
Name Version Rev Tracking Publisher Notes
alsa-utils 1.1.2-5 68 latest/stable canonical✓ -
bluez 5.48-4 314 latest/stable canonical✓ -
canonical-livepatch 10.8.3 282 latest/stable canonical✓ -
core 16-2.61.4-20240607 17200 latest/stable canonical✓ core
core18 20240612 2829 latest/stable canonical✓ base
core20 20240416 2318 latest/stable canonical✓ base
core22 20240408 1380 latest/stable canonical✓ base
htop 3.3.0 4407 latest/stable maxiberta
nextcloud 29.0.2snap1 42890 latest/stable nextcloud✓ -
pc 18-2 104 18/stable canonical✓ gadget
pc-kernel 4.15.0-225.237 1827 18/stable canonical✓ kernel
pulseaudio 8.0-3 9 latest/stable canonical✓ -
snapd 2.63 21759 latest/stable canonical✓ snapd
sysconfs-nuc7 ←←← was pre-installed with the image for NUC
snap info pc-kernel
name: pc-kernel
summary: The canonical generic amd64 kernel
publisher: Canonical✓
store-url: https://snapcraft.io/pc-kernel
contact: snaps@canonical.com
license: unset
description: |
The Ubuntu kernel package as a snap
type: kernel
snap-id: …
tracking: 18/stable
refresh-date: today at 16:02 UTC
Should I report this somewhere else? do you need any log?
Example of auto update bricking the server (I left my desktop terminal open to ssh into the server):
cm-t@Desktop-t:~$ ssh cm-t@1x.x.x.x
Welcome to Ubuntu Core 18 (GNU/Linux 4.15.0-225-generic x86_64)
Last login: Sat Jun 29 16:04:44 2024 from x.x.x.x
# a few moment later…
cm-t@localhost:~$
Broadcast message from root@localhost (Mon 2024-07-01 03:20:37 UTC):
reboot scheduled to update the system
The system is going down for reboot at Mon 2024-07-01 03:30:37 UTC!
Broadcast message from root@localhost (Mon 2024-07-01 03:20:40 UTC):
reboot scheduled to update the system
The system is going down for reboot at Mon 2024-07-01 03:21:40 UTC!
Broadcast message from root@localhost (Mon 2024-07-01 03:21:40 UTC):
reboot scheduled to update the system
The system is going down for reboot NOW!
Connection to x.x.x.x closed by remote host.
Connection to x.x.x.x closed.
cm-t@Desktop-t:~$
You can use snap refresh --hold to stop refreshes. Can you attach the information about the change? Run snap changes find out which failed and then attach the output of snap change <id>. Is there any output on the console?
cm-t@localhost:~$ snap changes
ID Status Spawn Ready Summary
342 Error 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Auto-refresh snap "pc-kernel"
343 Error yesterday at 07:51 UTC yesterday at 07:55 UTC Auto-refresh snap "pc-kernel"
344 Error yesterday at 17:10 UTC yesterday at 18:26 UTC Auto-refresh snap "pc-kernel"
345 Error yesterday at 18:31 UTC yesterday at 19:20 UTC Auto-refresh snap "pc-kernel"
346 Done today at 00:00 UTC today at 00:00 UTC Running service command for snap "nextcloud"
347 Done today at 00:00 UTC today at 00:00 UTC Running service command for snap "nextcloud"
348 Done today at 00:00 UTC today at 00:00 UTC Running service command for snap "nextcloud"
349 Done today at 00:00 UTC today at 00:00 UTC Running service command for snap "nextcloud"
350 Done today at 00:00 UTC today at 00:00 UTC Running service command for snap "nextcloud"
351 Done today at 00:00 UTC today at 00:00 UTC Running service command for snap "nextcloud"
352 Error today at 03:20 UTC today at 06:14 UTC Auto-refresh snap "pc-kernel"
353 Error today at 06:19 UTC today at 06:26 UTC Auto-refresh snap "pc-kernel"
cm-t@localhost:~$ snap change 342
Status Spawn Ready Summary
Done 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Ensure prerequisites for "pc-kernel" are available
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Download snap "pc-kernel" (1892) from channel "18/stable"
Done 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Fetch and check assertions for snap "pc-kernel" (1892)
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Mount snap "pc-kernel" (1892)
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Run pre-refresh hook of "pc-kernel" snap if present
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Stop snap "pc-kernel" services
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Remove aliases for snap "pc-kernel"
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Make current revision for snap "pc-kernel" unavailable
Done 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Update assets from kernel "pc-kernel" (1892)
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Copy snap "pc-kernel" data
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Setup snap "pc-kernel" (1892) security profiles
Undone 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Make snap "pc-kernel" (1892) available to the system
Error 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Automatically connect eligible plugs and slots of snap "pc-kernel"
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Set automatic aliases for snap "pc-kernel"
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Setup snap "pc-kernel" aliases
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Run post-refresh hook of "pc-kernel" snap if present
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Start snap "pc-kernel" (1892) services
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Remove data for snap "pc-kernel" (1691)
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Remove snap "pc-kernel" (1691) from the system
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Clean up "pc-kernel" (1892) install
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Run configure hook of "pc-kernel" snap if present
Hold 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Run health check of "pc-kernel" snap
Done 2 days ago, at 23:07 UTC yesterday at 07:46 UTC Monitoring snap "pc-kernel" to determine whether extra refresh steps are required
......................................................................
Update assets from kernel "pc-kernel" (1892)
2024-06-29T23:07:35Z INFO No gadget assets update needed
......................................................................
Make snap "pc-kernel" (1892) available to the system
2024-06-29T23:07:35Z INFO Task set to wait until a system restart allows to continue
......................................................................
Automatically connect eligible plugs and slots of snap "pc-kernel"
2024-06-30T07:46:50Z ERROR cannot finish pc-kernel installation, there was a rollback across reboot
......................................................................
Monitoring snap "pc-kernel" to determine whether extra refresh steps are required
2024-06-29T23:07:35Z INFO Task set to wait until a system restart allows to continue
cm-t@localhost:~$ snap change 343
Status Spawn Ready Summary
Done yesterday at 07:51 UTC yesterday at 07:55 UTC Ensure prerequisites for "pc-kernel" are available
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Download snap "pc-kernel" (1892) from channel "18/stable"
Done yesterday at 07:51 UTC yesterday at 07:55 UTC Fetch and check assertions for snap "pc-kernel" (1892)
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Mount snap "pc-kernel" (1892)
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Run pre-refresh hook of "pc-kernel" snap if present
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Stop snap "pc-kernel" services
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Remove aliases for snap "pc-kernel"
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Make current revision for snap "pc-kernel" unavailable
Done yesterday at 07:51 UTC yesterday at 07:55 UTC Update assets from kernel "pc-kernel" (1892)
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Copy snap "pc-kernel" data
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Setup snap "pc-kernel" (1892) security profiles
Undone yesterday at 07:51 UTC yesterday at 07:55 UTC Make snap "pc-kernel" (1892) available to the system
Error yesterday at 07:51 UTC yesterday at 07:55 UTC Automatically connect eligible plugs and slots of snap "pc-kernel"
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Set automatic aliases for snap "pc-kernel"
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Setup snap "pc-kernel" aliases
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Run post-refresh hook of "pc-kernel" snap if present
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Start snap "pc-kernel" (1892) services
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Remove data for snap "pc-kernel" (1691)
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Remove snap "pc-kernel" (1691) from the system
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Clean up "pc-kernel" (1892) install
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Run configure hook of "pc-kernel" snap if present
Hold yesterday at 07:51 UTC yesterday at 07:55 UTC Run health check of "pc-kernel" snap
Done yesterday at 07:51 UTC yesterday at 07:55 UTC Monitoring snap "pc-kernel" to determine whether extra refresh steps are required
......................................................................
Update assets from kernel "pc-kernel" (1892)
2024-06-30T07:52:03Z INFO No gadget assets update needed
......................................................................
Make snap "pc-kernel" (1892) available to the system
2024-06-30T07:52:08Z INFO Task set to wait until a system restart allows to continue
......................................................................
Automatically connect eligible plugs and slots of snap "pc-kernel"
2024-06-30T07:55:21Z ERROR cannot finish pc-kernel installation, there was a rollback across reboot
......................................................................
Monitoring snap "pc-kernel" to determine whether extra refresh steps are required
2024-06-30T07:52:08Z INFO Task set to wait until a system restart allows to continue
edit: i’ve set the snap refresh hold until there is a solution as you suggested
Thank you. Yes this indicates that the boot was rolled back as you observed and the change was reverted. I have notified some folks on the team and we’ll try to reproduce.
Did you get any chance to observe whether there was any output on the console maybe?
