I am new here, so please excuse if I am missing important information on this request.
I am using snapd on OpenSuse/Tumbleweed on my Dell notebook. Mainly to run Teams-For-Linux and Postman. Until two days ago everything was great. Then during a “zypper dup” snapd was upgraded to version 2.53.1:
mknoblau@c3b4m33lx:~> snap version
snap 2.53-1.3
snapd 2.53-1.3
series 16
opensuse-tumbleweed 20211011
kernel 5.13.7-1-default
Since then both teams-for-linux and postman do not start anymore. The small Hello-World snap works fine. Symptom of the failing snaps:
I am not sure whether this is an snapd issue, or something else that happened during the “zypper dup”, so I am a bit at a loss here. Folks at Tumbleweed are silent about it.
Any one seen this recently? Any hints? Any advice on how to debug this?
Thanks a lot in advance
Martin
PS: and of course I did that upgrade on both my working setups …
There’s something offa bout the kernel you’re running:
Whereas I have:
snap 2.53-1.4
snapd 2.53-1.4
series 16
opensuse-tumbleweed 20211012
kernel 5.14.9-1-default
Pretty sure the previous snapshot had 5.14.9 kernel too.
What arch is that? My openSUSE laptop is x86_64 and posman, ohmygiraffe, spotify all work fine. The CI testing is also done on the same architecture and none report problems. I’ve also tried to procure the conditions like you have, where there are no entries in the device map, and the apps start as usual.
I think I’d start by zypper dup and making sure that the kernel is updated. I only know of one distribution that uses 5.13 where cgroups v2 are enabled and devices are filtered using BPF, and it’s Ubuntu 21.10. I’m also assuming that the kernel was patched significantly. My other box which runs Arch, has 5.14.12, and Debian Sid in our testing is on 5.14.9.
The next thing to check is apparmor. On openSUSE the audit messages are actually consumed by auditd, so please attach the output of sudo ausearch -m AVC.
Although the error message suggest the problem may be elsewhere, and the error log obtained from the kernel is empty
thanks for coming back to me. The kernel on that laptop is self built. And actually I am now on 5.14-12 as well. The laptop is a Dell Precision 7540 (x86_64) - nothing exotic. The installation is up2date with the exception of the kernel.
mknoblau@c3b4m33lx:~> snap version
snap 2.53-1.4
snapd 2.53-1.4
series 16
opensuse-tumbleweed 20211012
kernel 5.14.12-1-default
mknoblau@c3b4m33lx:~> uname -a
Linux c3b4m33lx 5.14.12-1-default #1 SMP PREEMPT Thu Oct 14 09:21:30 CEST 2021 x86_64 x86_64 x86_64 GNU/Linux
So personally I doubt the kernel is at fault, but as a last resort I certainly can switch back to the distro kernel.
Let’s see if this may be a problem. Can you edit /etc/apparmor.d/usr.libexec.snapd.snap-confine, and add capability bpf in there? You can add it after capability sys_admin, so that the content is like:
OK, some new effects … So I added “bpf” and restarted apparmor. That made the denied for “bpf” go away, but no change in problem. Same after adding “net_admin”. No more denied messages, but postman/teams-for-linux still fail to start.
So I finally decided to install the current distro kernel from tumbleweed (5.14.9):
mknoblau@c3b4m33lx:~> snap version
snap 2.53-1.4
snapd 2.53-1.4
series 16
opensuse-tumbleweed 20211012
kernel 5.14.9-1-default
That made the difference. “postman” now comes up. So, something in my kernel configuration seems to be missing/odd. Is there anything that has to be enabled/defined for snap to run? Or should not be enabled/defined? If you have cycles, I am happy to send you my configuration.
Unfortunately this is not the end of the story, because “teams-for-linux” still fails (never failed before the system upgrad four days ago) with the following symptom:
mknoblau@c3b4m33lx:~> teams-for-linux
/snap/teams-for-linux/182/teams-for-linux: error while loading shared libraries: libxshmfence.so.1: cannot open shared object file: No such file or directory
That library is installed in /usr/lib64. And the weird thing: if I start he executable directly, it comes up !!!
So, one final last guess: remove all snaps and reinstall. Now it works !!!
Check if CONFIG_CGROUP_BPF=y is in your configuration, also a more general CONFIG_BPF=y and CONFIG_BPF_SYSCALL=0 although those are likely enabled. I suggest taking a look at https://build.opensuse.org/package/show/Kernel:stable/kernel-source specifically the config file and look for BPF and cgroups.
Those are different things. First, a library in /usr/lib64 on the host is not the same as that library being inside the snap. A snap runs inside its own mount namespace (think chroot on steroids). Secondly, AFAICT teams-for-linux is a strictly confined snap, so the fact that you ran the binary directly and it work is just an accident, as it’s not guaranteed to work this way (that’s what snaps are for), and it’s unconfined at this point. IIRC my kids use teams snap which is published directly by Microsoft Teams team.
so in the end it was CONFIG_CGROUP_BPF=y missing from my configuration. Adding it and rebuilding my kernel makes “teams-for-linux” and “postman” come up OK again. So in the end it was my own fault Really thanks a big lot for your support.
Why I had to reinstall the snaps is not clear to me. I repeated the kernel exercise on my second system and there it was not neccessary to reinstall the snaps. But I am not complaining.
So, just to conclude this, on the second system I checked the apparmor messages. There were some “DENIED”, but they do not seem to hurt:
