New experimental "snap run --experimental-gdbserver" option

snapd
1

The latest edge snapd build contains a new way to debug snaps. Unlike the current “snap run --gdb” this new option runs the application as the regular user and also allows attaching gdb frontends that can talk to gdbserver. It still requires sudo once for the attaching of gdbserver.

To debug a program you can use:

$ snap run --experimental-gdbserver test-snapd-tools-core18.echo "hello gdb"

Welcome to "snap run --gdbserver".
You are right before your application is run.
Please open a different terminal and run:

gdb -ex="target remote :38881" -ex=continue -ex="signal SIGCONT"
(gdb) continue

or use your favorite gdb frontend and connect to :38881

and on a different terminal:

$ gdb -ex="target remote :38881" -ex=continue -ex="signal SIGCONT"
GNU gdb (Ubuntu 9.1-0ubuntu1) 9.1
...
Reading target:/usr/lib/debug/lib/x86_64-linux-gnu//libc-2.27.so from remote target...
(No debugging symbols found in target:/lib/x86_64-linux-gnu/libc.so.6)
Reading symbols from target:/lib64/ld-linux-x86-64.so.2...
Reading /lib64/ld-2.27.so from remote target...
Reading /lib64/.debug/ld-2.27.so from remote target...
Reading /usr/lib/debug//lib64/ld-2.27.so from remote target...
Reading /usr/lib/debug/lib64//ld-2.27.so from remote target...
Reading target:/usr/lib/debug/lib64//ld-2.27.so from remote target...
(No debugging symbols found in target:/lib64/ld-linux-x86-64.so.2)
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
0x00007f58caef5e75 in raise () from target:/lib/x86_64-linux-gnu/libc.so.6
Continuing.

--Type <RET> for more, q to quit, c to continue without paging--
Program received signal SIGSTOP, Stopped (signal).
0x00007f58caef5e75 in raise () from target:/lib/x86_64-linux-gnu/libc.so.6
Continuing with signal SIGCONT.

Program received signal SIGCONT, Continued.
0x00007f58caef5e97 in raise () from target:/lib/x86_64-linux-gnu/libc.so.6
(gdb)

At this points it’s right before the exec() of the program. Using “cont” will run the program normally.

6 Likes
2

Cool!

Is there a plan to start building snaps with debug symbols by default? Or at least all the snaps provided directly by Canonical?

Also, is there a plan to capture and debug from a core file, if a snap program crashes?

1 Like
3

It seems that gdbserver is not in the core18 or core20 base snaps, so this cannot be used on Ubuntu Core 18 or 20 as is, however I tested just adding the gdbserver package to core20 and it just worked, and only increased the size of the snap by ~200K, so I will be proposing PR’s to those base snaps adding gdbserver so that this can be used with Ubuntu Core snaps for remote debugging.

Note that it appears gdbserver is already in the core snap, so this just works on UC16.

Edit: the related PR’s were merged so this should work correctly in UC18 and UC20

1 Like
4

We’ve added documentation for this feature (and gdb), which can now be found here: Using gdb and gdbserver.