ERROR cannot remove snap file "snapcraft"

After some time without using a machine, I entered a snap source tree and:

$ snapcraft clean --use-lxd
internal error, please report: running "snapcraft" failed: cannot find installed snap "snapcraft" at revision 9542: missing file /snap/snapcraft/9542/meta/snap.yaml

So I tried reinstalling Snapcraft:

$ sudo snap refresh snapcraft
snap "snapcraft" has no updates available
$ sudo snap refresh --edge snapcraft
2023-09-14T14:03:38Z ERROR cannot remove snap file "snapcraft", will retry in 3 mins: umount: /snap/snapcraft/9510: not mounted.
2023-09-14T14:06:38Z ERROR cannot remove snap file "snapcraft", will retry in 3 mins: umount: /snap/snapcraft/9510: not mounted.

Of course the error would go on forever — I actually let the session sit and it ran for 15 h straight with the same error.

Snapd is from the stable channel.

==> /proc/self/mountinfo <==
23 28 0:21 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
24 28 0:22 / /proc rw,nosuid,nodev,noexec,relatime shared:12 - proc proc rw
25 28 0:5 / /dev rw,nosuid,relatime shared:2 - devtmpfs udev rw,size=8161852k,nr_inodes=2040463,mode=755,inode64
26 25 0:23 / /dev/pts rw,nosuid,noexec,relatime shared:3 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
27 28 0:24 / /run rw,nosuid,nodev,noexec,relatime shared:5 - tmpfs tmpfs rw,size=1636684k,mode=755,inode64
28 1 252:1 / / rw,relatime shared:1 - ext4 /dev/vda1 rw,discard,errors=remount-ro,commit=30
29 23 0:6 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime shared:8 - securityfs securityfs rw
30 25 0:25 / /dev/shm rw,nosuid,nodev shared:4 - tmpfs tmpfs rw,inode64
31 27 0:26 / /run/lock rw,nosuid,nodev,noexec,relatime shared:6 - tmpfs tmpfs rw,size=5120k,inode64
32 23 0:27 / /sys/fs/cgroup rw,nosuid,nodev,noexec,relatime shared:9 - cgroup2 cgroup2 rw
33 23 0:28 / /sys/fs/pstore rw,nosuid,nodev,noexec,relatime shared:10 - pstore pstore rw
34 23 0:29 / /sys/fs/bpf rw,nosuid,nodev,noexec,relatime shared:11 - bpf bpf rw,mode=700
36 24 0:30 / /proc/sys/fs/binfmt_misc rw,relatime shared:13 - autofs systemd-1 rw,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22511
37 25 0:31 / /dev/hugepages rw,relatime shared:14 - hugetlbfs hugetlbfs rw,pagesize=2M
38 25 0:19 / /dev/mqueue rw,nosuid,nodev,noexec,relatime shared:15 - mqueue mqueue rw
39 23 0:7 / /sys/kernel/debug rw,nosuid,nodev,noexec,relatime shared:16 - debugfs debugfs rw
40 23 0:12 / /sys/kernel/tracing rw,nosuid,nodev,noexec,relatime shared:17 - tracefs tracefs rw
41 23 0:32 / /sys/fs/fuse/connections rw,nosuid,nodev,noexec,relatime shared:18 - fusectl fusectl rw
42 23 0:20 / /sys/kernel/config rw,nosuid,nodev,noexec,relatime shared:19 - configfs configfs rw
65 27 0:33 / /run/credentials/systemd-sysctl.service ro,nosuid,nodev,noexec,relatime shared:20 - ramfs ramfs rw,mode=700
67 27 0:34 / /run/credentials/systemd-sysusers.service ro,nosuid,nodev,noexec,relatime shared:21 - ramfs ramfs rw,mode=700
69 27 0:35 / /run/credentials/systemd-tmpfiles-setup-dev.service ro,nosuid,nodev,noexec,relatime shared:22 - ramfs ramfs rw,mode=700
97 28 252:15 / /boot/efi rw,relatime shared:48 - vfat /dev/vda15 rw,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro
100 28 7:1 / /snap/lxd/24918 ro,nodev,relatime shared:50 - squashfs /dev/loop1 ro,errors=continue,threads=single
135 27 0:36 / /run/credentials/systemd-tmpfiles-setup.service ro,nosuid,nodev,noexec,relatime shared:54 - ramfs ramfs rw,mode=700
106 36 0:37 / /proc/sys/fs/binfmt_misc rw,nosuid,nodev,noexec,relatime shared:56 - binfmt_misc binfmt_misc rw
313 27 0:24 /snapd/ns /run/snapd/ns rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=1636684k,mode=755,inode64
676 27 0:49 / /run/user/1000 rw,nosuid,nodev,relatime shared:189 - tmpfs tmpfs rw,size=1636680k,nr_inodes=409170,mode=700,uid=1000,gid=1000,inode64
548 28 7:3 / /snap/core20/1974 ro,nodev,relatime shared:188 - squashfs /dev/loop3 ro,errors=continue,threads=single
565 28 7:4 / /snap/snapcraft/9510 ro,nodev,relatime shared:295 - squashfs /dev/loop4 ro,errors=continue,threads=single
788 28 0:48 / /var/snap/lxd/common/ns rw,relatime - tmpfs tmpfs rw,size=1024k,mode=700,inode64
808 788 0:4 mnt:[4026532206] /var/snap/lxd/common/ns/shmounts rw - nsfs nsfs rw
767 788 0:4 mnt:[4026532258] /var/snap/lxd/common/ns/mntns rw - nsfs nsfs rw
1393 28 7:5 / /snap/snapd/19457 ro,nodev,relatime shared:378 - squashfs /dev/loop5 ro,errors=continue,threads=single
1494 28 7:6 / /snap/core22/817 ro,nodev,relatime shared:396 - squashfs /dev/loop6 ro,errors=continue,threads=single
314 28 7:7 / /snap/lxd/25112 ro,nodev,relatime shared:150 - squashfs /dev/loop7 ro,errors=continue,threads=single
365 313 0:4 mnt:[4026532203] /run/snapd/ns/lxd.mnt rw - nsfs nsfs rw
503 28 7:8 / /snap/core22/858 ro,nodev,relatime shared:184 - squashfs /dev/loop8 ro,errors=continue,threads=single
80 28 7:9 / /snap/snapcraft/9542 ro,nodev,relatime shared:32 - squashfs /dev/loop9 ro,errors=continue,threads=single
534 28 7:10 / /snap/core20/2015 ro,nodev,relatime shared:195 - squashfs /dev/loop10 ro,errors=continue,threads=single
714 28 7:11 / /snap/snapd/19993 ro,nodev,relatime shared:202 - squashfs /dev/loop11 ro,errors=continue,threads=single
111 27 0:38 / /run/credentials/systemd-resolved.service ro,nosuid,nodev,noexec,relatime shared:35 - ramfs ramfs rw,mode=700
47 28 252:1 /snap /snap rw,relatime shared:1 - ext4 /dev/vda1 rw,discard,errors=remount-ro,commit=30
60 47 7:12 / /snap/snapd/20092 ro,nodev,relatime shared:310 - squashfs /dev/loop12 ro,errors=continue,threads=single
63 28 7:12 / /snap/snapd/20092 ro,nodev,relatime shared:310 - squashfs /dev/loop12 ro,errors=continue,threads=single

How to proceed?

Hi nteodosio,

It looks like snapd state is referring to files that does not exist. Can you think of any manual actions that might have caused changes to snapd managed files?

Will let you know if I find a safe workaround to that to recommend.

Hi @ernestl,

I can’t think of any, the machine was left sitting for weeks before I came back to it and observed the problem.

Thanks for the help, I can easily reset that machine though, so if that is the simpler solution and there is no interest in an investigation, I can do it.

Hi nteodosio,

I am investigating further, will share my findings. Perhaps easiest for you for now to install fresh snapd or similar reset.