Problem with squashfs in basic ubuntu:18.04 docker image


#1

snapd isn’t starting up because squashfs isn’t working. I don’t understand the kernel well enough to make sense out of what Google turns up on this issue.

Further details

I’d like to use Snap in a Docker image, based on phusion/baseimage, which in turn is based on ubuntu:18.04. Phusion/baseimage uses the runit init system. When I “apt-get -qy install snapd squashfs-tools” in the docker file, and also set up /etc/service/snapd/run to execute snapd, I get these errors:

helpers.go:146: error trying to compare the snap system key: system-key missing on disk
daemon.go:379: started snapd/2.39.2+18.04 (series 16; classic; devmode) ubuntu/18.04 (amd64) linux/4.9.125-linuxkit.
main.go:123: system does not fully support snapd: cannot mount squashfs image using “squashfs”: mount: /tmp/sanity-mountpoint-588159470: mount failed: Operation not permitted.

Perhaps this will help

“cat /proc/filesystems” returns a lot of stuff, which includes squashfs.

“uname -a” returns
Linux 9416d2a723bf 4.9.125-linuxkit #1 SMP Fri Sep 7 08:20:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

“dpkg --list | grep linux-image” returns nothing

wget https://people.canonical.com/~john/test.squashfs
mount test.squashfs /mnt/squash/example
The above two lines complain: “mount: /mnt/squash/example: mount failed: Operation not permitted.”


#2

Are you running a privileged container? It’s likely that the default AppArmor profile for your docker container denies the ability to perform mounts, so even though you have squashfs available through the kernel, AppArmor is denying the mount.


#3

Thank you ijohnson, AppArmor was disabled. The symptoms change when I add the --privileged option to “docker run”. That gets me past the “Operation not permitted” to “wrong fs type, bad option, bad superblock on /tmp/sanity-squashfs-095778552, missing codepage or helper program, or other error.”

I see also a “snap system key: system-key missing” error.

AppArmor status: apparmor not enabled
helpers.go:146: error trying to compare the snap system key: system-key missing on disk
daemon.go:379: started snapd/2.39.2+18.04 (series 16; classic; devmode) ubuntu/18.04 (amd64) linux/4.9.125-linuxkit.
main.go:123: system does not fully support snapd: cannot mount squashfs image using “fuse.snapfuse”: mount: /tmp/sanity-mountpoint-125646327: wrong fs type, bad option, bad superblock on /tmp/sanity-squashfs-095778552, missing codepage or helper program, or other error.


#4

What do you see in the system journal inside the container?

Also, note that this setup of running snaps inside containers is very much not supported, so this is really a best effort thing and it’s entirely possible that this won’t work without a long list of hacks and other changes that you really don’t want to make to your docker containers.


#5

I notice I’m able to mount test.squashfs now that I’m using the --privileged option, so the problem seems be specific to snapd. I now think I need a bunch more stuff than snapd. Looking at the systemd configuration: ls /lib/systemd/system/snapd*
/lib/systemd/system/snapd.autoimport.service
/lib/systemd/system/snapd.snap-repair.service
/lib/systemd/system/snapd.core-fixup.service
/lib/systemd/system/snapd.snap-repair.timer
/lib/systemd/system/snapd.failure.service
/lib/systemd/system/snapd.socket
/lib/systemd/system/snapd.seeded.service
/lib/systemd/system/snapd.system-shutdown.service
/lib/systemd/system/snapd.service

I will try that in the morning.

I’m not certain what you mean by “system journal”. /var/log has a lot of stuff in it, and none of it seems to contain anything interesting - but maybe I’ve missed something. In particular /var/log/journal is empty. syslog-ng is being used.

ls /var/log
alternatives.log auth.log btmp dpkg.log fontconfig.log lastlog mongodb nginx redis sysstat wtmp
apt bootstrap.log cron.log faillog journal messages mysql postgresql syslog tallylog

I naively expected that containers would be no harder than IoT devices, which I think I’ve read Snap is supposed to support.


#6

Snaps do work on IoT devices and they are supported. However running snaps inside docker containers is not a supported workflow specifically because of the restrictions that are imposed on docker containers. Snaps do run inside LXD containers without much configuration because LXD containers are much closer to a full system setup inside a namespace, but docker containers are not setup the same way.


#7

You’re going to have a hard time if you’re trying to port snapd off of systemd. There are circumstances where snapd will exit and expect to be restarted the next time it is needed via socket activation.

And separate from snapd, installing a snap that provides a system daemon will cause a systemd service unit to be installed and started. That’s going to fail if you’ve ripped out systemd and replaced it with runit.


#8

Jamesh, snapd services that exit shouldn’t be a problem, runit automatically restarts services that quit.

I guess I could set up something to watch for new systemd services and add adapters for runit, but it’s not worth it. I will either abandon Snap or switch to systemd.


#9

IJohnson, it’s looking more like I should just drop Snap. Thanks!