Snapd STILL requires out-of-tree apparmor patches for strict confinement

Ubuntu is currently the only distribution, as far as I can tell, that supports strict confinement out-of-the-box. This is due to the fact that snapd relies on out-of-tree apparmor patches that are only applied in Ubuntu distributed kernels. Specifically, this patch is required to get snapd to support strict confinement, which has still not been upstreamed. Why, after 3 years of this being known (Snapd vs upstream kernel vs apparmor) are we still in this situation?

From reading about the upstreaming process for this particular patch it seems that it was not upstreamed on purpose due to it re-enabling legacy features. In that case then why are we relying on functionality that is not ever going to be upstreamed and claiming that we are fully functional on many distributions when that claim can only ever apply to Ubuntu?

7 Likes

Manjaro has strict confinement out of the box; I just checked this in a VM:

$ snap debug confinement
strict
$ snap debug sandbox-features
apparmor:             kernel:caps kernel:dbus kernel:domain kernel:file kernel:mount kernel:namespaces kernel:network kernel:network_v8 kernel:policy kernel:ptrace kernel:query kernel:rlimit kernel:signal parser:unsafe policy:default support-level:full
confinement-options:  classic devmode strict
dbus:                 mediated-bus-access
kmod:                 mediated-modprobe
mount:                freezer-cgroup-v1 layouts mount-namespace per-snap-persistency per-snap-profiles per-snap-updates per-snap-user-profiles stale-base-invalidation
seccomp:              bpf-actlog bpf-argument-filtering kernel:allow kernel:errno kernel:kill_process kernel:kill_thread kernel:log kernel:trace kernel:trap kernel:user_notif
udev:                 device-cgroup-v1 device-filtering tagging

I thought Solus had strict confinement too, but I don’t have a VM to check this. Ubuntu derivatives should also have strict confinement, but I’m guessing this is not what you mean.

@jdstrand posted a great summary of what is needed at the moment:

For the second one, we just need to wait until every distro gets the latest AppArmor. The first one isn’t upstreamed yet, @jdstrand explained why:

LSM stacking is a long way off, but AF_UNIX support might come sooner:

6 Likes

The second one isn’t part of any AppArmor userspace stable release yet so there is nothing distro can do.

AIUI, in addition to Manjaro and Ubuntu, quite a few distributions support strict confinement. OTOH some are Elementary, KDE Neon, Kubuntu, Mint, Lubuntu, Pop_OS!, Solus, Xubuntu and Zorin (a few are Ubuntu flavors, some derivatives that utilize the AppArmor patch set and others simply picked them up (they are self-contained after all)).

There have been updates since the update I gave not long ago. AppArmor 3 is in RC and will be uploaded to Ubuntu 20.10 (groovy) soon (days to week or two) with final coming before 20.10 is released. That obviously is only for one release of Ubuntu, but once that is done and we shake out bugs, etc, the idea I plan to propose is for snapd to vendor apparmor 3 sometime during the next development cycle (ie, sometime between November and April). This will allow using networkv8 that is in upstream kernels instead of requiring the network compat patchset.

Furthermore, we’ve reprioritized AF_UNIX wrt LSM stacking and are today actively working to land AF_UNIX in the upstream kernel ahead of the various LSM stacking iterations/pieces.

10 Likes

Two great decisions, thanks @jdstrand!

1 Like

fwiw, here is our situation:

  • we want to use snapd on a nvidia jetson device
  • we want to use yocto with the required nvidia L4T layers and kernel
  • the latest kernel that nvidia uses is 4.9

bummer =(

I’m wondering if https://ubuntu.com/blog/nvidia-canonical-expand-partnership-to-edge could help …?

3 Likes

I understand that Jamie is no longer working with Canonical. So could anyone from the Snapd OR Ubuntu Security team please clarify an update on the plan proposed by Jamie there ?

We currently patch the Linux kernel with these 3 patches in our snapd Yocto recipe, I am wondering when can we drop those.

4 Likes

@alexmurray - perhaps you can comment?

The progress of vendoring apparmor into snapd can be seen in https://github.com/snapcore/snapd/pull/9939 - this is pretty close but for now we are only targeting the snap build of snapd for this to avoid making too much work for downstream distributors of snapd etc.

The upstreaming of AF_UNIX is still ongoing - regarding those 3 patches, is the first one strictly required? It looks more like utility functions being added and I can’t see those being referenced by the other 2 so I suspect this may be able to be dropped. The second one is only needed if you are using apparmor 2.x in userspace - if you are using apparmor 3 then that should also be able to be dropped. This then leaves only the third one which adds the actual AF_UNIX support - this is still needed for now.

1 Like

If more patches except one aren’t needed then why all ubuntu releases including 21.10 devel still ship them? They have apparmor 3.0 userspace.

1 Like

Which is the current status of all this @alexmurray? (Apart from https://github.com/snapcore/snapd/pull/9939 which was already merged)

Thanks!

The changes to vendor apparmor into snapd have been merged but subsequently reverted twice now - unfortunately the complexity involved here is larger than initially realised - so that is still ongoing but I hope to have another stab at it again in near future.

As far as getting the various AppArmor kernel features merged upstream, that work is still ongoing as well. With any luck both should hopefully happen within the first few months of this year :crossed_fingers:

1 Like

Thanks for the info!

Is there anywhere where one can follow this upstream process?

@jjohansen manages all of the upstream kernel work - I am not sure if the git branches are all up to date but you can take a look at https://git.kernel.org/pub/scm/linux/kernel/git/jj/linux-apparmor.git/log/?h=apparmor-next which is where the most recent/pending upstream pull-request will be staged. Also the request to Linus is usually CC’d to the linux-security-module mailing list - https://lore.kernel.org/linux-security-module/ - ie. https://lore.kernel.org/linux-security-module/167105671974.22509.5991919513100894698.pr-tracker-bot@kernel.org/T/#m3f7509e4abe670bc496f6cb2e4ea416a0ae5f659

1 Like

This is coming in snapd 2.60, isn’t it?

2 Likes

Yes, as of snapd 2.60, the snapd snap has it’s own apparmor_parser and associated config etc and will use this in preference to the system installed one.

You can check this yourself by running snap debug sandbox-features and looking for the attribute parser:snapd-internal under the apparmor section.

[amurray:~] $ snap version
snap    2.60.1
snapd   2.60.1
series  16
ubuntu  23.04
kernel  6.2.0-24-generic
[amurray:~] $ snap debug sandbox-features
apparmor:             kernel:caps kernel:dbus kernel:domain kernel:file kernel:ipc kernel:mount kernel:namespaces kernel:network kernel:network_v8 kernel:policy kernel:ptrace kernel:query kernel:rlimit kernel:signal parser:cap-audit-read parser:cap-bpf parser:include-if-exists parser:mqueue parser:qipcrtr-socket parser:snapd-internal parser:unsafe parser:userns parser:xdp policy:default support-level:full
confinement-options:  classic devmode strict
dbus:                 mediated-bus-access
kmod:                 mediated-modprobe
mount:                layouts mount-namespace per-snap-persistency per-snap-profiles per-snap-updates per-snap-user-profiles stale-base-invalidation
seccomp:              bpf-actlog bpf-argument-filtering kernel:allow kernel:errno kernel:kill_process kernel:kill_thread kernel:log kernel:trace kernel:trap kernel:user_notif
udev:                 tagging
2 Likes

Does this mean that distros shipping snapd 6.20+ should be able to support strict confinement with the default kernel? Or is there any extra config that must be done first?

(I meant 2.60+, ahah) I decided to go and test this on a fresh Pop!_OS installation by installing snapd through the apt repos and subsequently installing snapd from the Snap Store, to get the latest version.

Here are the results:

test@pop-os:~$ snap version
snap    2.60.3
snapd   2.60.3
series  16
pop     22.04
kernel  6.5.4-76060504-generic
test@pop-os:~$ snap debug sandbox-features
apparmor:             kernel:caps kernel:domain kernel:file kernel:mount kernel:namespaces kernel:network_v8 kernel:policy kernel:ptrace kernel:query kernel:rlimit kernel:signal parser:cap-audit-read parser:cap-bpf parser:include-if-exists parser:mqueue parser:qipcrtr-socket parser:snapd-internal parser:unsafe parser:userns parser:xdp policy:default support-level:partial
confinement-options:  classic devmode
dbus:                 mediated-bus-access
kmod:                 mediated-modprobe
mount:                layouts mount-namespace per-snap-persistency per-snap-profiles per-snap-updates per-snap-user-profiles stale-base-invalidation
seccomp:              bpf-actlog bpf-argument-filtering kernel:allow kernel:errno kernel:kill_process kernel:kill_thread kernel:log kernel:trace kernel:trap kernel:user_notif
udev:                 tagging

The good news is that parser:snapd-internal does indeed appear. The bad news is that support-level:partial is still reported, with strict missing from confinement-options.

Unfortunately, I’m not extremely familiar with the inner workings of snapd, so if I may ask, what more might need to be done to ensure strict confinement is available on non-Ubuntu distros like Pop!_OS and openSUSE Tumbleweed?

The internal parser means that snapd uses the apparmor parser shipped with snapd, which brings all necessary apparmor userspace. Also debian bookworm also only has partial support with a 6.4 kernel.

The info I from the implementers is that only socket mediation is missing in the mainline kernel.

Getting the ubuntu source via apt source linux and grepping through it, it is still in the ubutu diff and not in the mainline source.

root@t--23-10:~/linux-source# grep -i AF_UNIX linux_6.5.0-7.7.diff  | grep mediation
+    - SAUCE: apparmor4.0.0 [06/76]: af_unix mediation

Side Note: dbus mediation was said to require AF_UNIX mediation. But that dbus mediation is listed in sandbox-features ‘dbus: mediated-bus-access’ since a while. I guess they found a way to do mediation without AF_UNIX support (or it is partial?).

One can compile their own kernel with the patch, but caveat emptor and i do not know how to do that in a well maintainable way (auto-patching/compiling new kernels) without needing a lot of infrastructure (own apt repositories, a CI/pipeline for compiling and publishing the modified kernel).

The patches were maintained in apparmor-kernel. As @jdstrand left canonical I am not sure who is carrying the torch for this topic. @jjohansen is working on LSM stacking, but from my outside view does not seem currently to be involved in AF_UNIX mediation topics.

It certainly would be good to know if anybody is actually working on this or if there are usable workarounds.

1 Like

I started this topic three years ago, and we’re still in the same boat that if people want to or need to use strict confinement, or provide strict confinement of snaps within their distros, then the distro or the user needs to have a non-upstream kernel or have manually patched it themselves.

It is really disappointing to me to see that we still need to use a modified kernel to support the snap ecosystem properly.

5 Likes