Security.nesting required to run snaps inside Incus containers?

snapd
1

Hi! I’ve just installed Incus 6.2 on a server of mine running Ubuntu 24.04 and found out that snaps were not running inside containers, because some devpts permission denied error. An example:

pachulo@server:~$ incus launch images:ubuntu/24.04 test-snaps
pachulo@server:~$ incus exec test-snaps -- su --login ubuntu
ubuntu@test-snaps:~$ sudo apt install snapd
ubuntu@test-snaps:~$ sudo snap install hello-world
ubuntu@test-snaps:~$ hello-world
cannot perform operation: mount -t devpts --make-slave --make-private -o acl,relatime,kernmount,iversion,active,nouser,0xffffffff00000000 devpts /dev/pts: Permission denied

I’ve asked in the Linux Containers forum and they suggested that I tried security.nesting=true on the container, which solved the problem.

So I was guessing: is this expected behaviour or a bug somewhere?

Thanks!

1 Like
3

Hello

Let me try to reproduce that locally.

$ sudo apt install -y incus
$ sudo incus admin init
$ incus launch images:ubuntu/24.04 test-snaps
$ incus exec test-snaps -- su --login ubuntu
$ sudo apt install -y snapd
$ snap version # (prints 2.66.1+24.04)
$ sudo snap install hello-world
$ hello-world

The version of snapd I got is:

ubuntu@test-snaps:~$ snap version
snap    2.66.1+24.04
snapd   2.66.1+24.04
series  16
ubuntu  24.04
kernel  6.8.0-49-generic

The failure was:

ubuntu@test-snaps:~$ hello-world
cannot perform operation: mount -t devpts --make-slave --make-private -o acl,relatime,kernmount,iversion,active,nouser,0xffffffff00000000 devpts /dev/pts: Permission denied

Now back on my outer host I got this failure:

zyga@wyzima:~$ sudo journalctl -n 1000 | grep DENIED
gru 11 10:06:31 wyzima kernel: audit: type=1400 audit(1733911591.717:347): apparmor="DENIED" operation="mount" class="mount" info="failed perms check" error=-13 profile="incus-test-snaps_</var/lib/incus>" name="/dev/pts/" pid=135295 comm="snap-confine" fstype="devpts" srcname="devpts"

This suggests that incus does not allow the container to mount devpts at /dev/pts/ . The failed perms check message implies that no such permission was around. Importantly the apparmor profile is generated by incus, not snapd, so we cannot do anything about it.

A quick look at incus data confirms this:

root@wyzima:/var/lib/incus/security/apparmor/profiles# grep 'fstype=' incus-test-snaps 
  mount fstype=binfmt_misc -> /proc/sys/fs/binfmt_misc/,
  mount fstype=configfs -> /sys/kernel/config/,
  mount fstype=debugfs -> /sys/kernel/debug/,
  mount fstype=efivarfs -> /sys/firmware/efi/efivars/,
  mount fstype=tracefs -> /sys/kernel/tracing/,
  mount fstype=fuse,
  mount fstype=fuse.*,
  mount fstype=fusectl -> /sys/fs/fuse/connections/,
  mount fstype=hugetlbfs,
  mount fstype=mqueue,
  mount fstype=proc -> /proc/,
  mount fstype=securityfs -> /sys/kernel/security/,
  mount fstype=sysfs -> /sys/,
  mount fstype=ramfs,
  mount fstype=tmpfs,
  mount fstype=cgroup -> /sys/fs/cgroup/**,
  mount fstype=cgroup2 -> /sys/fs/cgroup/**,

As we can see incus simply does not allow mounting devpts inside containers.

I cannot comment on the security aspects of running incus with some sort of privileged mode or enabling nesting.

4

Could we open a PR to the upstream perhaps? Does lxd allow this mount?