I’m building a snap for amd64, arm64, and armhf. The snap builds on all 3 architectures and runs on amd64 and arm64. However, I’m having trouble with armhf. I’ve tried using Raspberry Pi’s running Ubuntu 18 and 19.
This is a daemon app, so when I execute sudo snap start mysnap, I get the typical obscure daemon message:
error: cannot perform the following tasks:
- start of [mysnap.mydaemon] (# systemctl start snap.mysnap.mydaemon.service
Job for snap.mysnap.mydaemon.service failed because a fatal signal was delivered causing the control process to dump core.
See "systemctl status snap.mysnap.mydaemon.service" and "journalctl -xe" for details.
)
The odd part is that, when I try to run sudo snap run --shell mysnap.mydaemon to manually execute the
daemon wrapper, I get:
Illegal instruction
If I run without sudo, I get
Illegal instruction (core dumped)
The most informative error message comes from running sudo snap run --strace=--raw mysnap.mydaemon, which tracks early snap helpers. If I don’t use the --strace=--raw option, I get
error: signal: illegal instruction
When I run sudo snap run --strace=--raw mysnap.mydaemon, I get a long output ending in:
...
access("/var/lib/snapd/seccomp/bpf//snap.mysnap.mydaemon.bin", F_OK) = 0
stat64("/", {st_mode=S_IFDIR|0755, st_size=296, ...}) = 0
stat64("/var", {st_mode=S_IFDIR|0755, st_size=159, ...}) = 0
stat64("/var/lib", {st_mode=S_IFDIR|0755, st_size=220, ...}) = 0
stat64("/var/lib/snapd", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/var/lib/snapd/seccomp", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/var/lib/snapd/seccomp/bpf", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("/var/lib/snapd/seccomp/bpf/snap.mysnap.mydaemon.bin", {st_mode=S_IFREG|0644, st_size=32, ...}) = 0
openat(AT_FDCWD, "/var/lib/snapd/seccomp/bpf//snap.mysnap.mydaemon.bin", O_RDONLY|O_LARGEFILE) = 4
fstat64(4, {st_mode=S_IFREG|0644, st_size=32, ...}) = 0
read(4, " \0\0\0\4\0\0\0\25\0\0\1(\0\0@\6\0\0\0\1\0\5\0\6\0\0\0\0\0\0\0", 32768) = 32
read(4, "", 28672) = 0
close(4) = 0
seccomp(SECCOMP_SET_MODE_FILTER, SECCOMP_FILTER_FLAG_LOG, {len=4, filter=0x7e9a9ce8}) = 0
access("/var/lib/snapd/seccomp/bpf/global.bin", F_OK) = -1 EPERM (Operation not permitted)
umask(022) = 037777777777
readlink("/proc/self/fd/3", 0x7e9b0cf4, 4096) = -1 EPERM (Operation not permitted)
write(2, "cannot read symbolic link target"..., 48) = -1 EPERM (Operation not permitted)
write(2, ": Operation not permitted\n", 26) = -1 EPERM (Operation not permitted)
exit_group(1) = -1 EPERM (Operation not permitted)
exit(1) = -1 EPERM (Operation not permitted)
--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPC, si_addr=0x76e92b7e} ---
+++ killed by SIGILL (core dumped) +++
error: signal: illegal instruction
It seems to me like snapd is having trouble accessing the global.bin seccomp profile. Furthermore, this issue is limited to this snap, not even previous versions of this snap that are on the snapcraft store.
Is this a known issue? Are there any directions I should go in for debugging this?
I should mention that I’m having to build with snapcraft --use-lxd because these pi’s don’t support multipass’s use of kvm virtualization. I had to do the same with the arm64 version of the snap, although that was on a bare metal AWS ec2. This didn’t cause problems on arm64.
Updates
When I install the snap on devmode, I’m able to get the daemon running. I’ll investigate the possibility that this problem is a result of improper seccomp/apparmor profiles in the morning. I was going off the assumption that they’d be the same across architectures. This is odd because snappy-debug didn’t give any output.
