Troubleshooting snap-confine and apparmor on Gentoo

snapd
1

I’ve been using snaps on Gentoo by using JamesB192’s overlay on github. I wrote a little blog post about it when I first set it up on my system and it worked well.
A while ago it stopped working and I didn’t have time to look into it and I’ve now setup a new laptop and hit the same issue. I’m hoping someone here has a better understanding of apparmor and snaps can help me troubleshoot what’s missing on my system:

Main error:

snap run chromium
snap-confine has elevated permissions and is not confined but should be. Refusing to continue to avoid permission escalation attacks

$ snap version
snap    {2.36.1}
snapd   {2.36.1}
series  16
gentoo  -
kernel  4.20.6-gentoo

$ sudo aa-status
apparmor module is loaded.
54 profiles are loaded.
53 profiles are in enforce mode.
    /snap/core/6350/usr/lib/snapd/snap-confine
    /snap/core/6350/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
    snap-update-ns.chromium
    snap-update-ns.core
    snap-update-ns.powershell
    snap.chromium.chromium
    snap.core.hook.configure
    syslog-ng
    syslogd
    traceroute
1 profiles are in complain mode.
    snap.powershell.powershell
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

PS, I chopped some of the output from the above command (additional irrelevant apparmor profiles)

$ cat /usr/src/linux/.config|grep APPARMOR
CONFIG_SECURITY_APPARMOR=y
CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=1
CONFIG_SECURITY_APPARMOR_HASH=y
CONFIG_SECURITY_APPARMOR_HASH_DEFAULT=y
# CONFIG_SECURITY_APPARMOR_DEBUG is not set
# CONFIG_DEFAULT_SECURITY_APPARMOR is not set

$ cat /boot/grub/grub.cfg|grep apparmor|head -1
linux	/boot/vmlinuz-4.20.6-gentoo root=/dev/nvme0n1p6 ro init=/lib/systemd/systemd apparmor=1 security=apparmor quiet

Also the output of journalctl -g apparmor can be found here

Please let me know if you notice anything out on my configuration or other information that might help.

Many thanks

1 Like
2

While we wait for the people who actually know this bit of snapd, could you paste the output of snap debug confinement and snap debug sandbox-features?

Also, where did that funky version string come from? :slight_smile:

3

How is the version string funky? What is it supposed to look like?

snap debug confinement
partial

snap debug sandbox-features
apparmor:             kernel:caps kernel:domain kernel:file kernel:mount kernel:namespaces kernel:network_v8 kernel:policy kernel:ptrace kernel:query kernel:rlimit kernel:signal parser:unsafe policy:downgraded support-level:partial
confinement-options:  classic devmode
dbus:                 mediated-bus-access
kmod:                 mediated-modprobe
mount:                freezer-cgroup-v1 layouts mount-namespace per-snap-persistency per-snap-profiles per-snap-updates per-snap-user-profiles stale-base-invalidation
seccomp:              bpf-argument-filtering kernel:allow kernel:errno kernel:kill_process kernel:kill_thread kernel:log kernel:trace kernel:trap
udev:                 device-cgroup-v1 tagging
4 
$  snap version
snap    2.37.1
snapd   2.37.1
series  16
ubuntu  16.04
kernel  4.4.0-142-generic

look ma, no braces :slight_smile:

5

Hey! Thank you for sharing this. It is fantastic to see snaps used on Gentoo!

I will respond briefly as I’m unavailable this week.

I’m not familiar with the specific overlay packaging yet. It is likely that the overlay is not installing or enabling the snapd.apparmor.service file that is responsible for loading apparmor profiles specific to snapd.

6

Great tip thanks @zyga-snapd. I will check that out as soon as I can and post results.

7

Unless Gentoo is using reexec, there should be a profile from snap-confine from the host fs too. This is what I have on Arch:

maciek@galeon:~ sudo aa-status
[sudo] password for maciek: 
apparmor module is loaded.
227 profiles are loaded.
226 profiles are in enforce mode.
   ....
   /usr/lib/snapd/snap-confine
   /usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   ....
   /var/lib/snapd/snap/core/6259/usr/lib/snapd/snap-confine
   /var/lib/snapd/snap/core/6259/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /var/lib/snapd/snap/core/6350/usr/lib/snapd/snap-confine
   /var/lib/snapd/snap/core/6350/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
  ...

The profile is in /etc/apparmor.d. On Arch there is a apparmor.service that needs to be enabled at boot and the it loads all the profiles from /etc.

8

Thankyou @mborzecki. I think this is narrowing it down alot.
I fixed the incorrect version issue, so now:

$ snap version
snap    2.37.2
snapd   2.37.2
series  16
gentoo  -
kernel  4.20.6-gentoo

Also read a bit of the source code to find how it determines if it is confined. I must say, it is quite easy to read. I was able to enable debugging and get the following:

$ export SNAP_CONFINE_DEBUG=1
$ snap run chromium
DEBUG: security tag: snap.chromium.chromium
DEBUG: executable:   /usr/lib/snapd/snap-exec
DEBUG: confinement:  non-classic
DEBUG: base snap:    core18
DEBUG: ruid: 1000, euid: 0, suid: 0
DEBUG: rgid: 1000, egid: 0, sgid: 0
DEBUG: apparmor label on snap-confine is: unconfined
DEBUG: apparmor mode is: (null)
snap-confine has elevated permissions and is not confined but should be. Refusing to continue to avoid permission escalation attacks

A little searching on that debug info and I found this post.
That coupled with @mborzecki’s reply points me to the fact that there is no snap-confine profile in /etc/apparmor.d

I couldn’t find this file in the source, is it something that can be generated?

Many thanks

9

The profile is generated during the build provided ---enable-apparmor was passed to configure. You will probably need to tweak ebuild to add it.

10

An update. Still not working. But I now have the apparmor profile in /etc/apparmor.d.
It seems to be loaded:

# aa-status
apparmor module is loaded.
56 profiles are loaded.
55 profiles are in enforce mode.
   /snap/core/6350/usr/lib/snapd/snap-confine
   /snap/core/6350/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   ....
   /usr/lib/snapd/snap-confine
   /usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   ....
   snap-update-ns.core
   snap-update-ns.notepad-plus-plus
   snap-update-ns.powershell
   snap.core.hook.configure
   snap.notepad-plus-plus.notepad-plus-plus
   ....
1 profiles are in complain mode.
   snap.powershell.powershell
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

Services all running:

$ sudo systemctl status apparmor
● apparmor.service - AppArmor profiles
   Loaded: loaded (/lib/systemd/system/apparmor.service; enabled; vendor preset: disabled)
   Active: active (exited) since Thu 2019-02-07 19:51:50 -00; 24min ago
 Main PID: 19793 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4915)
   Memory: 0B
   CGroup: /system.slice/apparmor.service

Feb 07 19:51:49 19wbpf2 systemd[1]: Starting AppArmor profiles...
Feb 07 19:51:50 19wbpf2 systemd[1]: Started AppArmor profiles.

$ sudo systemctl status snapd
● snapd.service - Snappy daemon
   Loaded: loaded (/lib/systemd/system/snapd.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-02-07 19:51:55 -00; 24min ago
 Main PID: 19837 (snapd)
    Tasks: 17 (limit: 4915)
   Memory: 7.7M
   CGroup: /system.slice/snapd.service
           └─19837 /usr/lib/snapd/snapd

Feb 07 19:51:55 19wbpf2 systemd[1]: Starting Snappy daemon...
Feb 07 19:51:55 19wbpf2 snapd[19837]: AppArmor status: apparmor is enabled but some kernel features are missing: dbus, network
Feb 07 19:51:55 19wbpf2 snapd[19837]: helpers.go:145: error trying to compare the snap system key: executable does not contain a build ID
Feb 07 19:51:55 19wbpf2 snapd[19837]: helpers.go:211: cannot write system key: executable does not contain a build ID
Feb 07 19:51:55 19wbpf2 snapd[19837]: daemon.go:379: started snapd/2.37.2 (series 16; classic; devmode) gentoo/ (amd64) linux/4.20.6-gentoo.
Feb 07 19:51:55 19wbpf2 systemd[1]: Started Snappy daemon.

$ sudo systemctl status snapd.apparmor
● snapd.apparmor.service - Load AppArmor profiles managed internally by snapd
   Loaded: loaded (/lib/systemd/system/snapd.apparmor.service; enabled; vendor preset: disabled)
   Active: active (exited) since Thu 2019-02-07 19:51:59 -00; 24min ago
 Main PID: 19896 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4915)
   Memory: 0B
   CGroup: /system.slice/snapd.apparmor.service

Feb 07 19:51:59 19wbpf2 systemd[1]: Starting Load AppArmor profiles managed internally by snapd...
Feb 07 19:51:59 19wbpf2 systemd[1]: Started Load AppArmor profiles managed internally by snapd.

Also a listing of the snapd apparmor profiles

$ ls /var/lib/snapd/apparmor/snap-confine/
$ ls /var/lib/snapd/apparmor/profiles/
snap-confine.core.6350    snap.notepad-plus-plus.notepad-plus-plus  snap-update-ns.core               snap-update-ns.powershell
snap.core.hook.configure  snap.powershell.powershell                snap-update-ns.notepad-plus-plus

Not sure if there was supposed to be something in the snap-confine directory.
Anyway, I’m still getting the same error as in the last message and I’m out of ideas for now. If anyone can make a suggestion or give me a hint on where to look next it would be greatly appreciated.

Thanks

11

Update. Pretty much cracked this and as usual it’s such a simple issue when you see it. On Gentoo without multilib, /usr/lib is a symlink to /usr/lib64, so when I generate the profile, I need to take symlinks into consideration.

I haven’t got it fully working yet because I need to fix all the other assumptions in the profile to include /usr/lib{,64} where it supported, but I’ll do a full write up when I’m done.

12

PS, it was extremely obvious the issue, when I ran aa-enforce /usr/lib/snapd/snap-confine. It complains that a profile doesn’t exist for /usr/lib64/snapd/snap-confine.

13

Tested out a few apps. Only one not working is Chromium:

main.go:224: cannot change mount namespace of snap "chromium"  according to change mount (/snap/gtk-common-themes/818/share/icons/Suru /snap/chromium/566/data-dir/icons/Suru none bind,ro 0 0): cannot use "/snap/gtk-common-themes/818/share/icons/Suru" as bind-mount source: not a directory
execl failed: No such file or directory

Searching for this came up with results from about march last year that was patched with a change to core. Anyone give any insight into this one?

14

Okay it seems to be an open issue: https://gitlab.gnome.org/Community/Ubuntu/gtk-common-themes/issues/1

So is this broken for 4 months and no-one cares?

15

The mount error is unrelated. IIRC this is fixed in gtk-common-themes from edge.

Can you install strace and do snap run --strace chromium ?

16

I will escalate this issue today.

17

@zyga-snapd no need to escalate. Worked it out.
Thankyou both @mborzecki.

Although running with --strace didn’t yield any additional output (maybe snap couldn’t find my strace?)
Running strace -s8192 -e file -f /snap/bin/chromium did and showed me this crucial detail:

[pid 28774] access("/usr/lib/snapd/snap-device-helper", X_OK) = -1 ENOENT (No such file or directory)

Yeah, that file didn’t exist.
Updated my ebuild to include that file and now Chromium is launching.

Very happy with the result.
I’m going to see if I can get some more seasoned Gentoo developers to look over my ebuild to make it align with their best practices but I wonder if you could use some of it to update zyga/gentoo-snappy ?

That way anyone looking from Gentoo can find it easier?

PS, you can find my ebuild here

Cheers

1 Like
18

Hey.

That’s amazing news. I pretty much neglected the gentoo-snappy repo as I was way too busy to maintain everything and there were few active users. If you are willing to help on ongoing maintenance we can definitely make it properly supported in snapd.

Can you please grab me on IRC in #snappy to discuss this?

19

I just released updates for 2.37.3 to a number of distributions. I would be willing to work on reviewing the gentoo overlay.

20

I am trying to get it work on funtoo/gentoo, no luck.
It always says AppArmor status: apparmor is enabled but some kernel features are missing: dbus, network

ls /sys/kernel/security/apparmor/features/
capability caps domain file mount namespaces network_v8 policy ptrace query rlimit signal

seems to be missing.

aa-status
apparmor module is loaded.
46 profiles are loaded.
46 profiles are in enforce mode.
/usr/lib/apache2/mpm-prefork/apache2
/usr/lib/apache2/mpm-prefork/apache2//DEFAULT_URI
/usr/lib/apache2/mpm-prefork/apache2//HANDLING_UNTRUSTED_INPUT
/usr/lib/apache2/mpm-prefork/apache2//phpsysinfo
/usr/lib/dovecot/anvil
/usr/lib/dovecot/auth
/usr/lib/dovecot/config
/usr/lib/dovecot/deliver
/usr/lib/dovecot/dict
/usr/lib/dovecot/dovecot-auth
/usr/lib/dovecot/dovecot-lda
/usr/lib/dovecot/dovecot-lda///usr/{bin,sbin}/sendmail
/usr/lib/dovecot/imap
/usr/lib/dovecot/imap-login
/usr/lib/dovecot/lmtp
/usr/lib/dovecot/log
/usr/lib/dovecot/managesieve
/usr/lib/dovecot/managesieve-login
/usr/lib/dovecot/pop3
/usr/lib/dovecot/pop3-login
/usr/lib/dovecot/ssl-params
/usr/lib/dovecot/stats
/usr/{bin,sbin}/apache2
/usr/{bin,sbin}/apache2//DEFAULT_URI
/usr/{bin,sbin}/apache2//HANDLING_UNTRUSTED_INPUT
/usr/{bin,sbin}/apache2//phpsysinfo
/usr/{bin,sbin}/avahi-daemon
/usr/{bin,sbin}/dovecot
/usr/{bin,sbin}/identd
/usr/{bin,sbin}/mdnsd
/usr/{bin,sbin}/nmbd
/usr/{bin,sbin}/nscd
/usr/{bin,sbin}/smbd
/usr/{bin,sbin}/smbldap-useradd
/usr/{bin,sbin}/smbldap-useradd///etc/init.d/nscd
/usr/{bin,sbin}/winbindd
/usr/{bin,sbin}/{,open}ntpd
dnsmasq
dnsmasq//libvirt_leaseshelper
klogd
nvidia_modprobe
nvidia_modprobe//kmod
ping
syslog-ng
syslogd
traceroute
0 profiles are in complain mode.
1 processes have profiles defined.
1 processes are in enforce mode.
/usr/sbin/ntpd (3000) /usr/{bin,sbin}/{,open}ntpd
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

I am out of idea, i will appreciate any help.