I would like to request enhanced debug capability with gdb for snaps.
In the current form, it is very difficult to analyze application runtime, for several reasons:
There is no automatic core dump when one is created.
If one manually sets a core_pattern under /proc/sys (even inside $SNAP writable area), there will be an apparmor denial.
Gdb runs as root, so the reproduction of errors inside the debugger is not identical to what the user may experience when running the snap.
I would request:
Ability to dump cores into an area that is accessible by the snap (but possibly not other snaps) in the context of the current user. Perhaps this would not be a default, but enabled with a separate flag, e.g.: --core-dump.
Ability to run gdb as the user.
Ability to load previously dumped cores in the context of the snap and the user.
Typically, for stripped binaries, you can do this:
objcopy --add-gnu-debuglink=foo.dbg foo
If the developer has symbols available, then could then link them during debug time, perhaps a combination of --gdb and --symbols option, which would then specify the location of symbols, and the actual linking would be done in the background (objcopy could be part of a symbols interface or similar).
This needs some design since we don’t want snaps modifying core_pattern, etc. I suspect we would want a crash handler ala apport/whoopsie that could process dumps from snaps and put them in the proper location. This should probably be accompanied with a ‘snap set’ command for turning off/on, defaulting to off. If that is in place, it shouldn’t be too difficult to add a companion command to ‘snap run --gdb’ to read the core file from the snap. I’m not sure that the core file should be in an existing SNAP_* location or in /var/cache/snapd/snap.<name>.* or similar.
A while back, I was working on a PR to get Snapcraft to do this:
This detached the debug symbols for binaries built by snapcraft itself (i.e. it didn’t handle stuff pulled in by stage-packages), and arranged them in a /usr/lib/debug style hierarchy indexed by build ID. Pointing GDB at these symbols via its debug-file-directory setting was enough to have it find symbols for the snapped binaries.
It’s obviously not a complete solution: collecting the symbols is just the first step towards being able to analyse crashes generated out in the field. It’s a necessary first step though.