How do I get service logs of a failing ot install snap

1

Hello,

I am packaging a service (GOGS) which should create a log file in $SNAP_COMMON directory. Service fails to start and I would like to know why by checking its log file, but it is removed as part failed snap clean up (I guess). Probably I can put a breakpoint at snapd code and pause it in debug mode, but is there a better/simpler way?

I am running this test in a build server and would like to have a more automated way of getting service log files. Also unfortunately this service does not log to syslog which would have been preserved.

Something like snap install --preserve-snap_common-for-debug app_name :slight_smile:

Thank you.

2

There are two places these logs might be available. In the system journal and in the changes list.

System Journal

journalctl -xe

Use the arrow keys to navigate…

Snapd Changes List

sudo snap changes

find the change which relates to the failed install and then:

sudo snap change $change_number

replace $change_number with the actual number of the change you want to inspect or set a variable with the change’s ID like below:

change_number=1234
sudo snap change $change_number
3

I actually know where log files are, they are in SNAP_COMMON/logs as I put them there. Usually this is fine and I can see logs.

My problem is that when I test changes in a snap service and install it on test server (no previous versions). Install process uncovers bugs and as a result install fails and also removes all the files for that app from SNAP_COMMON. Which is all good except that I cannot easily understand where is the bug by checking logs in SNAP_COMMON/logs as they are removed by snap failure cleanup (am I right?).

I actually have a workaround by sending logs to console which are sent to syslog and preserved even after snap cleanup.

But what will I do with a service which can only write logs to a file?

4

If your app must use a file, you can tell it to write into /dev/stdout or /dev/stderr

5

Sadly, this doesn’t seem to work in a strictly confined snap. Apparmor doesn’t let you access those path (ie any of /dev/std{out, err} and /proc/self/fd/*)

I’d really like this so I can direct nginx access logs to be part of the its unit logs in journald, but no dice.

6

I’m glad you have a workaround by outputting to the console.

As /dev/stdin, /dev/stdout and /dev/stderr are all symlinks to /proc/self/fd/{0,1,2} respectively, a rule like owner @{PROC}/@{pid}/{,task/[0-9]*/}fd/[0-9]* rw, needed to be investigated since the rule alone looks like it would allow access to other snap’s fd via /proc for anything running under the same uid. Upon investigation, the kernel ptrace LSM hooks are triggered with /proc/pid/fd access from another task. Put another way, the aforementioned rule does not allow access to other snap’s fds because we don’t have a corresponding AppArmor ptrace rule to allow that access. The above rule does allow a process to access its own fds via /proc.

This will be addressed in https://github.com/snapcore/snapd/pull/5189.

1 Like