Gnome-3-34 extension and Python segfault

I’m packaging a Python GTK 3 application. Currently I’m using the gnome-3-28 extension and I’m trying to change to gnome-3-34.

snapcraft.yaml

The program just crashes with segmentation fault. I can’t get GDB working in Snap.
Edit: Found snap run --gdb ...

Backtrace (not very helpful):

#0  0x0000000000000000 in ?? ()
#1  0x00007fddfdf4ddae in ffi_call_unix64 () from target:/snap/video-downloader/x25/usr/lib/x86_64-linux-gnu/libffi.so.6
#2  0x00007fddfdf4d71f in ffi_call () from target:/snap/video-downloader/x25/usr/lib/x86_64-linux-gnu/libffi.so.6
#3  0x00007fddfe953e6a in ?? () from target:/snap/video-downloader/x25/gnome-platform/usr/lib/python3/dist-packages/gi/_gi.cpython-36m-x86_64-linux-gnu.so
#4  0x00007fddfe955a48 in ?? () from target:/snap/video-downloader/x25/gnome-platform/usr/lib/python3/dist-packages/gi/_gi.cpython-36m-x86_64-linux-gnu.so
#5  0x00000000005a9eec in _PyObject_FastCallKeywords ()
#6  0x000000000050a783 in ?? ()
#7  0x000000000050c1f4 in _PyEval_EvalFrameDefault ()
#8  0x0000000000507f24 in ?? ()
#9  0x0000000000516449 in ?? ()
#10 0x00000000005671ce in PyCFunction_Call ()

Another problem is that I want to install the ffmpeg package but it installs libraries (as dependencies) that are incompatible with libraries from the Gnome extension. This causes the build of the part libhandy to fail and causes problems when running the Snap because incompatible libraries are loaded.
Workaround: Build ffmpeg from source.

1 Like

While stage-packages are useful, they do have the limitation that whatever comes from outside of the archive would need to be binary compatible with those. This seems to be the problem here and there is not much that can be done. The alternative is to stick to gnome-3-28 unless there is a real good reason to migrate from it, at which point you face the tradeoff.

1 Like

This is why I think the desktop team are doing a disservice by promoting newer desktop stacks of Ubuntu-provided libraries without also providing recompiled libraries of every single other library that depends on the desktop stack. The gnome-3-34 specifically is a good example of why this is a bad idea: any snap that wants to use the gnome-3-34 extension has to explicitly only use the libraries in the content-snap or libraries that they compile from source. That snap must not use any other library from Ubuntu for fear that it wants a different dependency version than in the content snap. This is a terrible developer experience!

2 Likes

I’m on my phone right now, so I can’t easily compile a list but in the last few months, we have had a number of users with the same issue. This makes me think we should keep recommending the gnome-3-28 extension if the app doesn’t require a newer GTK version. (Afaik, this is the case right now. The GTK3 tutorial still uses gnome-3-28.)

2 Likes

Finally got it working: snapcraft.yaml

The solution was to compile PyGObject from source, the version bundled with the Gnome extension doesn’t work for whatever reason. Not even when using Python and libraries from the extension.

The app needs a current version of libhandy.

That’s my general experience with snapcraft.

Hi, I am sorry you feel that way, is there anything you think that could be done better?
While we cannot solve for multiple streams of binaries just working, certain things can be done about specific use cases you might have that we may not be aware of.

@kenvandine or @hellsworth mind looking into this?

I think this is a case that probably needs gnome-3-34. Libhandy wouldn’t build against the stage packages in gnome-3-28. I’m going to try to debug the PyGObject issue.

It looks like the problem is related to a different python3 at build time vs runtime. I’ve submitted a PR to fix this at https://github.com/Unrud/video-downloader/pull/38

Ultimately this should be fixed in the gnome-3-34 extension of snapcraft, which I’ve filed a bug for https://bugs.launchpad.net/snapcraft/+bug/1893262

2 Likes

1

My biggest problem is the buggy caching of already build parts and steps in snapcraft. It just causes random problems while tinkering with snapcraft.yaml and rebuilding it for testing.

Example: I changed the version number of PyGObject in the following part and for certain versions the build fails with the error that wheel and setuptools couldn’t be installed.

  pygobject:
    plugin: python
    python-packages:
      - pycairo==1.19.1
      - PyGObject==3.34.0

After trying to debug the problem I realized that running snapcraft clean cleared the issue. I previously just tried snapcraft clean pygobject because that normally fixes similar problems.

Example: While testing the PR from @kenvandine I removed the line build-environment: *buildenv from some modules and apparently that doesn’t cause a rebuild and snapcraft just continues to use the old cached result. (Take this example with a grain of salt because I haven’t looked into it too much and the issue might be something else.)

I ran into many similar problems that I can’t reproduce. Obviously this makes working with snapcraft painful because results of testing are basically random. The only way to be sure seems to be running snapcraft clean but that takes a long time.

2

snapcraft builds are not reproducible and additionally random versions of dependencies are pulled from the internet. That just adds to the previous problem.

3

An annoyance is that paths change between compile time and runtime. This requires patching installed programs or setting environment variables for programs that can’t find resources that were located by the buildsystem when compiling.

I guess (2) and (3) are a side effect of snapcraft’s design and can’t be fixed.

4

In general there are many rough edges that require workarounds.

It works but I don’t quite understand why. I think I tested similar things. I currently don’t want to waste even more time trying to understand what exactly is going on. Thanks for looking into it.

I don’t think the following does anything:

    environment:
      PATH: $SNAP_DESKTOP_RUNTIME/usr/bin:$PATH

See the result of the following command:

$ snap run --shell video-downloader -c 'type python3'
python3 is /snap/video-downloader/x3/usr/bin/python3

$ snap run --shell video-downloader -c 'echo "$PATH"'
/snap/video-downloader/x3/usr/sbin:/snap/video-downloader/x3/usr/bin:/snap/video-downloader/x3/sbin:/snap/video-downloader/x3/bin:/snap/video-downloader/x3/gnome-platform/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/video-downloader/x3/gnome-platform/usr/bin

Edit: I’ve build the Snap without it and it still works.

Setting PYTHONPATH shouldn’t be necessary when actually using Python from the extension, right?

    environment:
      ...
      PYTHONPATH: $PYTHONPATH:$SNAP_DESKTOP_RUNTIME/usr/lib/python3.6/site-packages:$SNAP/usr/lib/python3.6/site-packages:$SNAP/lib/python3.6/site-packages

I was under the impression that during build Python from the Gnome extension is preferred anyway. I might be remembering it wrong.

    build-environment: &buildenv
      - PATH: /snap/gnome-3-34-1804-sdk/current/usr/bin:$PATH
1 Like

The following does the trick:

    environment:
      PYTHONPATH: $SNAP_DESKTOP_RUNTIME/usr/lib/python3.6/site-packages

I think that I have figured it out:

  • The Gnome extensions comes with Python from the debian package
  • The debian version of Python is patched to load modules from usr/lib/python3*/dist-packages instead of usr/lib/python3*/site-packages.
  • The Gnome extension has a version of PyGObject in usr/lib/python3/dist-packages . This version doesn’t work and causes the SEGFAULT. I guess it’s from the python3-gi debian package.
  • The Gnome extension also has a version of PyGObject in usr/lib/python3.6/site-packages . That’s the correct version that was actually build for Gnome 3.34.

This explains why just running with $SNAP_DESKTOP_RUNTIME/usr/bin/python3 doesn’t work. It loads the module from dist-packages and not site-packages .
I only tested setting PYTHONPATH="$SNAP_DESKTOP_RUNTIME/usr/lib/python3/dist-packages" to get it working with $SNAP/usr/bin/python3 . I didn’t know that dist-packages is debian specific.

2 Likes

Why was my question and the solution flagged as spam?
I can’t find anything in the community guidelines that I obviously did wrong.