Snappy-debug improvements

Security policy and sandboxing discusses in the Debugging section the snappy-debug command (this command is discussed elsewhere too) and it says (in essence) to use snappy-debug.security scanlog like so:

$ sudo snappy-debug.security scanlog
...

I always found the invocation ‘sudo snappy-debug.security scanlog’ to be cumbersome, but the snappy-debug snap is quite old and it came from a time when it was thought that there would be a single debug snap that had things like gdb, strace, valgrind, scanlog, etc. While the invocation made some sense at the time, things have changed a lot and now we have snap run --strace ... and snap run --gdb ... as well as separate valgrind snaps such that snappy-debug has only contained the single scanlog functionality for years.

In the process of updating snappy-debug for 2.36, I decided to improve its invocation. Don’t worry though, the old documented sudo snappy-debug.security is unchanged and the invocation of sudo snappy-debug.security scanlog continues to work how it always has. With today’s stable update though, just invoke with snappy-debug, like so:

$ sudo snappy-debug
INFO: following '/var/log/syslog'. If have dropped messages, use:
INFO: $ sudo journalctl --output=short --follow --all | sudo snappy-debug
kernel.printk_ratelimit = 0
= AppArmor =
Time: Dec 17 15:54:48
Log: apparmor="DENIED" operation="mknod" profile="snap.hello-world.evil" name="/var/tmp/myevil.txt" pid=31749 comm="evil" requested_mask="c" denied_mask="c" fsuid=1000 ouid=1000
File: /var/tmp/myevil.txt (write)
Suggestion:
* adjust program to use TMPDIR or /tmp
...

As with the previous command, this also can be invoked with --only-snap=<name> to limit the output to that of a specific snap.

In addition, the snap now also exposes the underlying ‘scanlog’ command which, if one is so inclined, can be used to finetune results even further. See snappy-debug.scanlog --help for details.

Enjoy!

1 Like

I would like to ask if there’s a way to filter certain apparmor denials, like the following one caused by Flatpak:

[27979.160820] audit: type=1400 audit(1545116127.598:381): apparmor="DENIED" operation="open" profile="snap.my-awesome-app.my-app-name" name="/home/buo-ren/.local/share/flatpak/exports/share/icons/" pid=24311 comm="zenity" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000

Not currently, but its an interesting use case. I’ve added it to backlog.

1 Like

I made a few small but useful improvements today that I think @reviewers and snapd developers might enjoy for when a forum topic or bug comes in and only has raw seccomp denials from the audit subsystem of the kernel instead of snappy-debug formatted output:

$ snappy-debug.audit-arch 
Audit     Seccomp     Snapcraft   
c00000b7  aarch64     arm64
40000028  arm         armhf
00000014  ppc         powerpc
80000015  ppc64       --
c0000015  ppc64le     ppc64el
80000016  s390x       s390x
40000003  x86         i386
c000003e  x86_64      amd64

Also made the list searchable. Eg, say someone posts this:

audit: type=1326 audit(1478795610.347:140822): auid=4294967295 uid=1000 gid=1000 ses=4294967295 pid=16228 comm="ld-linux.so.2" exe="/snap/snap-ls/x1/lib/i386-linux-gnu/ld-2.23.so" sig=31 arch=40000003 syscall=195 compat=1 ip=0x565f765b code=0x0

What is arch=40000003?

$ snappy-debug.audit-arch 40000003
Audit     Seccomp     Snapcraft   
40000003  x86         i386

and what is syscall=195 for this arch?

$ snappy-debug.scmp-sys-resolver -a x86 195
stat64

scanlog has also been adjusted to use audit-arch instead of looking at the system’s arch, so now you can also simply pipe the raw log entry into snappy-debug.scanlog and get the correctly resolved syscall for the arch in the log ala (optionally with --recommend):

$ echo 'audit: type=1326 audit(1478795610.347:140822): auid=4294967295 uid=1000 gid=1000 ses=4294967295 pid=16228 comm="testme" exe="/testme.exe" sig=31 arch=c00000b7 syscall=219 compat=0 ip=0x565f765b code=0x0' | snappy-debug.scanlog 
= Seccomp =
Time: audit: type=132
Log: auid=4294967295 uid=1000 gid=1000 ses=4294967295 pid=16228 comm="testme" exe="/testme.exe" sig=31 arch=c00000b7 219(keyctl) compat=0 ip=0x565f765b code=0x0
Syscall: keyctl

Enjoy!

2 Likes

Please see the new --exclude option.

1 Like

In looking at another forum topic, it occurred to me that people may not know that if all you have is a log message, you can pipe that into snappy-debug for (hopefully!) meaningful results:

$ echo 'May 08 09:47:50 hostname kernel: audit: type=1400 audit(1588949270.087:1885): apparmor="DENIED" operation="open" profile="snap.hello-world.sh" name="/etc/fstab" pid=1474317 comm="cat" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0' | snappy-debug
= AppArmor =
Time: May 08 09:47:50
Log: apparmor="DENIED" operation="open" profile="snap.hello-world.sh" name="/etc/fstab" pid=1474317 comm="cat" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0
File: /etc/fstab (read)
Suggestions:
* adjust program to read necessary files from $SNAP, $SNAP_DATA, $SNAP_COMMON, $SNAP_USER_DATA or $SNAP_USER_COMMON
* adjust snap to use snap layouts (https://forum.snapcraft.io/t/snap-layouts/7207)
* add 'mount-observe' to 'plugs'

and with today’s edge, you can pipe AVC-style log denials as well:

$ echo 'May 08 09:47:50 hostname audit[1474317]: AVC apparmor="DENIED" operation="open" profile="snap.hello-world.sh" name="/etc/fstab" pid=1474317 comm="cat" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0' | snappy-debug
= AppArmor =
Time: May 08 09:47:50
Log: apparmor="DENIED" operation="open" profile="snap.hello-world.sh" name="/etc/fstab" pid=1474317 comm="cat" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0
File: /etc/fstab (read)
Suggestions:
* adjust program to read necessary files from $SNAP, $SNAP_DATA, $SNAP_COMMON, $SNAP_USER_DATA or $SNAP_USER_COMMON
* adjust snap to use snap layouts (https://forum.snapcraft.io/t/snap-layouts/7207)
* add 'mount-observe' to 'plugs'

This also works for seccomp denials and of course if you have a lot of denials, you can put them in a file and pipe it into snappy-debug too. Enjoy!

2 Likes

That is super useful, thanks @jdstrand!