A more graceful way for snapd to fail when /snap is missing

So i have move /snap to another disk because it was starting to take up too much space (don’t feel upset I did the same with flatpaks as well) however snap throw a wobbly and drops me into an emergency shell stopping the boot process if it start before the /snap mount is active. If I then manually mount it and continue the boot everything is fine. I have arranged the fstab order to put it just after boot but that did not help. So now I have had to resort to tweaking snapd.service file to force it to wait until the mount is complete.

So I don’t think I am the only one who wants to keep ./snap on a different disk so can we make it fail more gracefully when that disk is not mounted?


Instead of an fstab entry you should try a systemd mount unit instead and make it use:


That should get the dir mounted early enough …

Fairy nuff. I added a Requires=local-fs.target to snapd.service But making a separate systemd.mount unit would be more elegant, permanent solution.


That’s interesting. There’s nothing on /snap but a bunch of mountpoints and symlinks; you’re not saving much space at all. Is it really /snap?

That’s interesting. There’s nothing on /snap but a bunch of mountpoints and symlinks; you’re not saving much space at all. Is it really /snap?

Yes there is. And seeing as I use and NVME drive for root/home too much I feel.

du -xh /snap/ |tail 
4.0K	/snap/ultimate-media-downloader/3/usr/share/themes/Raleigh
3.0M	/snap/ultimate-media-downloader/3/usr/share/themes
66M	/snap/ultimate-media-downloader/3/usr/share
380M	/snap/ultimate-media-downloader/3/usr
0	/snap/ultimate-media-downloader/3/var/lib/ucf
0	/snap/ultimate-media-downloader/3/var/lib
0	/snap/ultimate-media-downloader/3/var
387M	/snap/ultimate-media-downloader/3
387M	/snap/ultimate-media-downloader
5.3G	/snap/

that’s very unexpected and broken and wrong. How did you get there? There shouldn’t be anything there beyond mountpoints and symlinks, e.g.

~$ du -xah /snap/ | sort -h
0       /snap/bin/go
0       /snap/bin/gofmt
0       /snap/bin/go.gofmt
0       /snap/bin/http
0       /snap/bin/http.man
0       /snap/bin/xbill-xaw
0       /snap/core/current
0       /snap/go/current
0       /snap/http/current
0       /snap/xbill-xaw/current
4.0K    /snap/bin
4.0K    /snap/core
4.0K    /snap/go
4.0K    /snap/http
4.0K    /snap/xbill-xaw
24K     /snap/

Indeed it looks like some content was unpacked into /snap by hand. snapd itself does not do that. Any normal system should look like what @chipaca describes above.

If you did not do that yourself, I’m quite curious to hear which tool did that.

As far as I know it was snapd that did that. All I have ever done is snap install and it dumps stuff in /snap with lots of symlinks at runtime.

I think maybe it is on a application level? These are my currently installed snaps.

snap list
Name                       Version        Rev   Developer       Notes
castersoundboard           0+git.c41a4f2  34    rikai           -
core                       16-2.28.5      3247  canonical       core
doctl                      1.7.0          13    digitalocean    -
krita                      3.3.1          27    krita           -
lonewolf                   19             32    mterry          -
magic-device-tool          1.0            286   marius-quabeck  devmode
pulsemixer                 1.3.0          1     snapcrafters    -
snapweb                    0.26-11-dev    307   canonical       -
teleconsole                0.3.1          3     popey           classic
uappexplorer-cli           1.0.0          2     bhdouglass      -
ultimate-media-downloader  6.0            3     keshavnrj       -

Maybe someone can install some of those packages and see where those archives end up ?

Edit: Oh FTR comparing my /snap to @chipaca

 126M	/snap/core/3017/usr/lib
126M	/snap/core/3247/usr/lib
207M	/snap/core/2898/usr
215M	/snap/core/3017/usr
215M	/snap/core/3247/usr
223M	/snap/magic-device-tool/286/usr
230M	/snap/magic-device-tool
230M	/snap/magic-device-tool/286
288M	/snap/ultimate-media-downloader/3/usr/lib/x86_64-linux-gnu
289M	/snap/core/2898
297M	/snap/core/3017
297M	/snap/core/3247
305M	/snap/ultimate-media-downloader/3/usr/lib
350M	/snap/krita/25/usr/lib/x86_64-linux-gnu
351M	/snap/krita/25/usr/lib
351M	/snap/krita/26/usr/lib
351M	/snap/krita/26/usr/lib/x86_64-linux-gnu
351M	/snap/krita/27/usr/lib/x86_64-linux-gnu
352M	/snap/krita/27/usr/lib
363M	/snap/castersoundboard/21/usr/lib/x86_64-linux-gnu
363M	/snap/castersoundboard/34/usr/lib/x86_64-linux-gnu
380M	/snap/ultimate-media-downloader/3/usr
387M	/snap/ultimate-media-downloader
387M	/snap/ultimate-media-downloader/3
389M	/snap/lonewolf/32/usr/lib/x86_64-linux-gnu
473M	/snap/krita/25/usr
474M	/snap/krita/26/usr
474M	/snap/krita/27/usr
481M	/snap/krita/25
482M	/snap/krita/26
482M	/snap/krita/27
487M	/snap/castersoundboard/21/usr/lib
487M	/snap/castersoundboard/34/usr/lib
513M	/snap/lonewolf/32/usr/lib
568M	/snap/castersoundboard/21/usr
569M	/snap/castersoundboard/34/usr
573M	/snap/castersoundboard/21
574M	/snap/castersoundboard/34
603M	/snap/lonewolf/32/usr
606M	/snap/lonewolf
606M	/snap/lonewolf/32
883M	/snap/core
1.2G	/snap/castersoundboard
1.5G	/snap/krita
4.8G	/snap/

Again, snapd never copies content into those directories as displayed above. There’s just no code doing that, anywhere. These are mountpoints only.

Given your original question:

Was that move done by recursively copying all the contents? That would indeed create the situation described above, and probably make snapd unhappy about working with those directories because they are supposed to be mountpoints, not trees of files.

Was that move done by recursively copying all the contents?

It was literally an rsync -av --partial --progress /snap .

and if if they were just symlinks then remove /snap should not have freed up 5+Gb of space should it? But it did. Furthermore installing new packages should have symlinks in my mounted /snap.

ls -la /snap
lrwxrwxrwx 1 root root 21 Oct 29 13:03 /snap -> /mnt/containers/snap/

ls -la /snap/
total 0
drwxr-xr-x 14 root root 229 Nov 13 16:15 .
drwxr-xr-x 4 root root 127 Nov 5 23:05 …
drwxr-xr-x 2 root root 221 Nov 13 16:15 bin
drwxr-xr-x 4 root root 41 Oct 28 17:01 castersoundboard
drwxr-xr-x 5 root root 57 Oct 28 17:03 core
drwxr-xr-x 3 root root 31 Sep 13 11:30 doctl
drwxr-xr-x 5 root root 51 Oct 10 12:04 krita
drwxr-xr-x 3 root root 31 Sep 6 13:48 lonewolf
drwxr-xr-x 3 root root 32 Sep 15 15:59 magic-device-tool
drwxr-xr-x 3 root root 30 Oct 10 23:04 pulsemixer
drwxr-xr-x 4 root root 43 Nov 13 16:13 snapweb
drwxr-xr-x 4 root root 39 Oct 3 10:07 teleconsole
drwxr-xr-x 3 root root 30 Oct 18 13:00 uappexplorer-cli
drwxr-xr-x 3 root root 30 Sep 24 16:23 ultimate-media-downloader

I just refreshed snapweb to the edge channel because it was showing up as broken in the stable version.

Well, if you ever find out who created those files in your filesystem please let us know. It wasn’t snapd for sure.

I suggest immediately undoing all of that by deleting all the content in /snap, creating the directory empty again in the root filesystem, and reinstalling the respective snaps. Or just remove the snapd package and reinstall it, making sure you have no /snap after removing the package, because snapd will not kill the content it doesn’t own there.

Then have a look at the result. It will look like the situation @chipaca described: symlinks and mountpoints only.

Note that if you are low on space and have an internal (mounted a boot) disk you can just mount a partition on /var/lib/snapd/snaps and copy the snaps over. This is where the space is really used.

Note that if you are low on space and have an internal (mounted a boot) disk you can just mount a partition on /var/lib/snapd/snaps and copy the snaps over. This is where the space is really used.

Ok now it becomes officially weird.

ls -la /var/lib/snapd/snaps/
total 1446636
drwxr-xr-x  3 root root      4096 Nov 13 16:15 .
drwxr-xr-x 15 root root      4096 Nov 13 16:20 ..
-rw-r--r--  1 root root 122626048 Aug 30 16:18 castersoundboard_21.snap
-rw-r--r--  1 root root 122671104 Sep  5 22:42 castersoundboard_34.snap
-rw-r--r--  1 root root  85348352 Sep 19 03:21 core_2898.snap
-rw-r--r--  1 root root  87080960 Oct 10 18:34 core_3017.snap
-rw-r--r--  1 root root  87089152 Oct 23 20:43 core_3247.snap
-rw-r--r--  1 root root   3350528 Sep 13 11:30 doctl_13.snap
-rw-r--r--  1 root root 114561024 Sep  6 00:39 krita_25.snap
-rw-r--r--  1 root root 114900992 Sep  6 00:33 krita_26.snap
-rw-r--r--  1 root root 114946048 Oct 10 12:04 krita_27.snap
-rw-r--r--  1 root root 130473984 Sep  6 13:48 lonewolf_32.snap
-rw-r--r--  1 root root  63033344 Sep 15 15:59 magic-device-tool_286.snap
drwxr-xr-x  2 root root      4096 Sep 26 18:41 partial
-rw-r--r--  1 root root   9011200 Oct 10 23:04 pulsemixer_1.snap
-rw-r--r--  1 root root 311910400 Nov 10 23:47 shotcut_15.snap
-rw-r--r--  1 root root   5074944 Nov 11 14:17 snapweb_300.snap
-rw-r--r--  1 root root   5074944 Nov 13 16:13 snapweb_307.snap
-rw-r--r--  1 root root   3616768 Sep 23 11:30 teleconsole_1.snap
-rw-r--r--  1 root root   3661824 Oct  3 10:07 teleconsole_3.snap
-rw-r--r--  1 root root  10481664 Oct 18 13:00 uappexplorer-cli_2.snap
-rw-r--r--  1 root root  86364160 Sep 24 16:23 ultimate-media-downloader_3.snap

du -h  /var/lib/snapd/snaps/
4.0K	/var/lib/snapd/snaps/partial
1.4G	/var/lib/snapd/snaps/

So 1.4Gb of snaps takes up 4.3Gb of real space?

and I am officially confused.

Snaps are heavily compressed. When you look at them they are uncompressed in /snap. When you look at the .snap files you see the real thing.

If you did that command on that target disk (/mnt/containers/snap/) then you definitely doubled up (likely even more because the target will have the data uncompressed) the space your snaps use by copying the content of the mounted compressed squashfs’es out of the mountpoints into the target disk …

Ok I guess I have had enough of this. I will blow away snapd completely and start again. This time looking for where it puts actual snaps and where is looback mounts them. Because I swear it was freeing up space by removing /snap

the kernel is clever enough and exposes the content of the mounted squashfs files, thats not actually the real disk space used :wink:

Exactly the opposite: 4.3GB of “real space” takes up 1.4GB of disk space!

Nice, isn’t it?


@veritanuda you have copied the contents of snap files to another disk. Not the snaps themselves. This won’t end well.

The disk space consumed by snaps is all in /var/lib/snapd/snaps, not /snaps

alan@hal:~$ du -hs /var/lib/snapd/snaps
16G	/var/lib/snapd/snaps

When you ‘snap install’ something, you get a .snap file in /var/lib/snapd/snaps/ - e.g. for teleconsole I have

 alan@hal:~$ ls -l /var/lib/snapd/snaps/teleconsole_3.snap 
-rw-r--r-- 1 root root 3661824 Oct  3 04:04 /var/lib/snapd/snaps/teleconsole_3.snap

That is the only disk space the snap takes up. That compressed file is mounted (not unpacked) in /snap/teleconsole/ and a symlink at /snap/teleconsole/current points to that

alan@hal:~$ mount | grep teleconsole
/var/lib/snapd/snaps/teleconsole_3.snap on /snap/teleconsole/3 type squashfs (ro,nodev,relatime)
alan@hal:~$ ls -l /snap/teleconsole/current
lrwxrwxrwx 1 root root 1 Oct  3 04:04 /snap/teleconsole/current -> 3

At runtime, yes, you will see files in /snap/teleconsole/3

alan@hal:~$ du -hs /snap/teleconsole/3/
15M	/snap/teleconsole/3/

However, if you shut the machine down, and examine the disk, you’ll see nothing in /snap, because they only get mounted when the system is started.

So what you’ve done is copy files from the insides of those .snap files to an external/separate disk. The snaps which get updated in /var/lib/snapd/snaps will have new and updated files which you will not have in your copies. You’ve basically broken the snap installs.

Yeah well I purged snapd now, made symlink from /var/lib/snapd/snaps -> /mnt/containers/snap and am installing warsone2100 now see where it all ends up. If as you say all the magic is in /var/lib/snapd/snaps then it should not take up any more space on root but on the /mnt/containers/snap instead. In which case my snapd.mount services is still needed to stop snapd from breaking boot but it just points to a different symlink then.

Hey ho.