PyGObject and Snap Problem

snapcraft
1

Hello!

I am trying to snap my python application which uses PyGObject for GTK3.

An older, working version (tkinter) of the app can be found here: https://snapcraft.io/ocrmypdfgui

I wrote a newer and more modern version that uses GTK3. Somehow I cannot get snapcraft to yield a running snap (build process works).

All code is here:

My snapcraft.yaml:

The snap crashes on run with:

alexander@alexander-XPS-13-9350:~/Programming/git/ocrmypdfgui$ ocrmypdfgui

** (main.py:76983): WARNING **: 23:40:23.003: Failed to load shared library ‘libgtk-3.so.0’ referenced by the typelib: /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3.so.0: undefined symbol: pango_font_description_set_variations Traceback (most recent call last): File “/snap/ocrmypdfgui/x1/lib/python3.6/site-packages/ocrmypdfgui/main.py”, line 3, in from ocrmypdfgui.gui import run File “/snap/ocrmypdfgui/x1/lib/python3.6/site-packages/ocrmypdfgui/gui.py”, line 6, in from ocrmypdfgui.ocr import start_job File “/snap/ocrmypdfgui/x1/lib/python3.6/site-packages/ocrmypdfgui/ocr.py”, line 12, in from gi.repository import Gtk, Gio, GLib File “”, line 971, in _find_and_load File “”, line 955, in _find_and_load_unlocked File “”, line 656, in _load_unlocked File “”, line 626, in _load_backward_compatible File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/importer.py”, line 146, in load_module dynamic_module = load_overrides(introspection_module) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/overrides/init.py”, line 118, in load_overrides override_mod = importlib.import_module(override_package_name) File “/snap/ocrmypdfgui/x1/usr/lib/python3.6/importlib/init.py”, line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/overrides/Gtk.py”, line 136, in class Widget(Gtk.Widget): File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/module.py”, line 176, in getattr interfaces = tuple(interface for interface in get_interfaces_for_object(info) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/module.py”, line 102, in get_interfaces_for_object interfaces.append(getattr(module, name)) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/overrides/init.py”, line 32, in getattr return getattr(self.introspection_module, name) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/module.py”, line 215, in getattr wrapper = metaclass(name, bases, dict) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/types.py”, line 250, in init register_interface_info(cls.info.get_g_type()) TypeError: must be an interface

Not sure how to fix this. Any ideas would be very appreciated.

1 Like
2

Is there a copy of libpango-1.0.so.0 in your snap? If there is, it is probably older than the one in the gnome-3-34 platform snap, which makes it incompatible with the platform snap’s libgtk-3.so.0.

If this is what’s happening, it’s likely due to something you’ve pulled in via stage-packages. You can likely work around this by removing the library from the list of files to be staged in your part:

parts:
  ocrmypdfgui:
    ...
    stage:
     - -usr/lib/*/libpango*

It’s possible that you’ve ended up staging other libraries provided by the platform snap, so you might be able to remove even more files.

3

I tried your suggestion which changed the error message somewhat:

alexander@alexander-XPS-13-9350:~/Programming/git/ocrmypdfgui$ ocrmypdfgui /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3-0/gtk-query-immodules-3.0: symbol lookup error: /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libpango-1.0.so.0: undefined symbol: hb_buffer_set_invisible_glyph ERROR: /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3-0/gtk-query-immodules-3.0 exited abnormally with status 127 g_module_open() failed for /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.so: /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libpango-1.0.so.0: undefined symbol: hb_buffer_set_invisible_glyph

** (process:22716): WARNING **: 10:38:01.743: Failed to load shared library ‘libpango-1.0.so.0’ referenced by the typelib: /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libpango-1.0.so.0: undefined symbol: hb_buffer_set_invisible_glyph Traceback (most recent call last): File “/snap/ocrmypdfgui/x1/lib/python3.6/site-packages/ocrmypdfgui/main.py”, line 3, in from ocrmypdfgui.gui import run File “/snap/ocrmypdfgui/x1/lib/python3.6/site-packages/ocrmypdfgui/gui.py”, line 6, in from ocrmypdfgui.ocr import start_job File “/snap/ocrmypdfgui/x1/lib/python3.6/site-packages/ocrmypdfgui/ocr.py”, line 12, in from gi.repository import Gtk, Gio, GLib File “”, line 971, in _find_and_load File “”, line 955, in _find_and_load_unlocked File “”, line 656, in _load_unlocked File “”, line 626, in _load_backward_compatible File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/importer.py”, line 145, in load_module importlib.import_module(‘gi.repository.’ + dep.split(“-”)[0]) File “/snap/ocrmypdfgui/x1/usr/lib/python3.6/importlib/init.py”, line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “”, line 994, in _gcd_import File “”, line 971, in _find_and_load File “”, line 955, in _find_and_load_unlocked File “”, line 656, in _load_unlocked File “”, line 626, in _load_backward_compatible File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/importer.py”, line 145, in load_module importlib.import_module(‘gi.repository.’ + dep.split(“-”)[0]) File “/snap/ocrmypdfgui/x1/usr/lib/python3.6/importlib/init.py”, line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “”, line 994, in _gcd_import File “”, line 971, in _find_and_load File “”, line 955, in _find_and_load_unlocked File “”, line 656, in _load_unlocked File “”, line 626, in _load_backward_compatible File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/importer.py”, line 146, in load_module dynamic_module = load_overrides(introspection_module) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/overrides/init.py”, line 118, in load_overrides override_mod = importlib.import_module(override_package_name) File “/snap/ocrmypdfgui/x1/usr/lib/python3.6/importlib/init.py”, line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/overrides/Pango.py”, line 41, in FontDescription = override(FontDescription) File “/snap/ocrmypdfgui/x1/gnome-platform/usr/lib/python3.6/site-packages/gi/overrides/init.py”, line 195, in override assert g_type != TYPE_NONE AssertionError

reads to me as if its now missing libpango.

In the build before - yes libpango-1.0.so is in the snap.

4

If I add:

python3-gi, python3-gi-cairo

to the stage packages. The libpango error changes to:

alexander@alexander-XPS-13-9350:~/Programming/git/ocrmypdfgui$ ocrmypdfgui /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3-0/gtk-query-immodules-3.0: symbol lookup error: /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3.so.0: undefined symbol: pango_font_description_set_variations ERROR: /snap/ocrmypdfgui/x1/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3-0/gtk-query-immodules-3.0 exited abnormally with status 127 Segmentation fault (core dumped)

Is this the infamous Gnome Extensions vs. Python Plugin Problem discussed here:

?

Will try building on Core20 with newer gnome extension…

5

core20 and gnome-3-38 seem to have real issues with python…

I cannot even build the snap anymore on this base because:

Skipping prime ocrmypdfgui (already ran)
Failed to generate snap metadata: The specified command 'python3 $SNAP/lib/python3.8/site-packages/ocrmypdfgui/__main__.py' defined in the app 'ocrmypdfgui' does not exist.
Ensure that 'python3 $SNAP/lib/python3.8/site-packages/ocrmypdfgui/__main__.py' is installed with the correct path

When I enter the snap with “snapcraft --debug” I can’t even find my installed python modules in parts/orcmypdfgui/ or in stage or prime…

Perhaps I am running into:

If I create a PIP .whl file I can install it locally just fine.

6

That’s really unfortunate, because the gnone-3-38 extension is ideal for use with PyGObject. For my most recent snap, I ended up using a custom installation of python 3.9 on core20 and hacking snapcraft.yaml to make it all work.

7

Do you have any idea why it doesn’t seem to correctly install my python project inside the snap? Perhaps pygobject is actually working fine with core20 und gnome-3-38.

Unfortunately I am no longer getting that far…

How did you make it work?

8

Have a look at this snapcraft.yaml snippet based on a snap I developed recently (I removed some irrelevant lines). Pay attention to the comments, particularly to the ones related to the python interpreter path, as this setup requires using the correct interpreter/venv.

parts:
  myapp:
    plugin: python
    source: .
    build-environment:
      - SNAPCRAFT_PYTHON_INTERPRETER: python3.9
      # The gnome-3-38 ext sets both env variables to point to its own Python v3.8, so we need to fix them manually.
      # $SNAPCRAFT_PART_INSTALL/bin is where the python3.9 executable lives.
      - PATH: $SNAPCRAFT_PART_INSTALL/bin:$PATH
      - PYTHONPATH: ''

    build-packages:
      - python3.9-venv
      # PyGObject dependencies (e.g. pycairo)
      - libcairo2-dev
      - python3.9-dev

    override-build: |
      # Work around a bug in snapcraft python plugin
      # https://forum.snapcraft.io/t/build-a-snap-with-any-version-of-python-i-want/10420/8
      rm -rf $SNAPCRAFT_PART_INSTALL/usr/lib/python3.9/distutils
      cp -r /usr/lib/python3.9/distutils $SNAPCRAFT_PART_INSTALL/usr/lib/python3.9/distutils
      mkdir -p $SNAPCRAFT_PART_INSTALL/usr/include
      cp -r /usr/include/python3.9 $SNAPCRAFT_PART_INSTALL/usr/include/python3.9
      snapcraftctl build
      
    stage-packages:
      - python3.9-venv

apps:
  myapp:
    # provides various desktop-related lib deps and env settings we need (PulseAudio, GObject introspection, etc.)
    extensions: [gnome-3-38]
    # Must be bin/python3.9 (or bin/python3), see logs at the end of the myapp part build (symlink creation)
    # Note: usr/bin/python3.9 does not work!
    command: bin/python3.9 -m myapp

And here’s a snippet of my setup.py for context:

from setuptools import setup, find_packages

setup(
    name='myapp',
    packages=find_packages(),

    # PyGObject needs pycairo downloaded and present before PyGObject can be installed, or the install will fail on the
    # first attempt. More details about setup_requires: https://setuptools.pypa.io/en/latest/userguide/keywords.html
    setup_requires=['pycairo'],

    install_requires=['PyGObject']
)
9

Ok… do you think that the python project not getting installed correctly is due to incorrect environment settings during build?

10

One small note: it is quite hard to read the logs you’re pasting. The easiest way to make them display correctly on this forum is to add a line containing three back ticks (i.e. ```) before and after the text. This will cause it to be treated as preformatted text.

The undefined symbol: hb_buffer_set_invisible_glyph error indicates you’ve got an old copy of libharfbuzz shadowing the version from the platform snap. You can exclude it in the same way as you did for pango.

Rather than going through these one-by-one, I’d suggest comparing the contents of your snap to the libraries in /snap/gnome-3-34-1804/current/usr/lib/x86_64-linux-gnu (the platform snap you’re using). If you notice any duplicates, exclude them from your snap.

11

try using a cleanup:part to remove all duplicated libraries (will likely also make your snap significantly smaller):

12

So I went back to base18 and gnome-3-34 for now.

I am trying to implement the Cleanup part.

Snapcraft.yaml:

name: ocrmypdfgui # you probably want to 'snapcraft register <name>'
title: ocrmypdfgui
base: core18 # the base snap is the execution environment for this snap
version: '0.9.04' # just for humans, typically '1.2+git' or '1.3.2'
summary: Hobby Project GUI for the Python Program "OCRmyPDF" by James R. Barlow
description: |
  I use James R. Barlow's OCRmyPDF heavily in my paperless Office and have created this Python Project as a GUI wrapper to run batch jobs on my filesystem. This is strictly a Hobby Project and is not "official". Feel free to use it if you like. Includes full current version of OCRmyPDF as backend. Icons and banner made by Freepik from www.flaticon.com
icon: gui/ocrmypdfgui.png
grade: devel # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

architectures: [amd64]

environment:
    TESSDATA_PREFIX: $SNAP/usr/share/tesseract-ocr/4.00/tessdata
    GS_LIB: $SNAP/usr/share/ghostscript/9.26/Resource/Init
    GS_FONTPATH: $SNAP/usr/share/ghostscript/9.26/Resource/Font
    LD_LIBRARY_PATH: $SNAP/usr/lib/x86_64-linux-gnu

apps:
  ocrmypdfgui:
    command: python3 $SNAP/lib/python3.6/site-packages/ocrmypdfgui/__main__.py #python3 $SNAP/lib/python3.6/site-packages/ocrmypdfgui/__main__.py
    desktop: $SNAPCRAFT_PROJECT_DIR/gui/ocrmypdfgui.desktop
    extensions: [gnome-3-34]

    plugs:
      - desktop
      - desktop-legacy
      - wayland
      - x11
      - home
      - removable-media


parts:
  ocrmypdfgui:
    # See 'snapcraft plugins'
    plugin: python
    source: .
    stage-packages: [ghostscript, icc-profiles-free, liblept5, libxml2, pngquant, tesseract-ocr-all, unpaper, qpdf, zlib1g]

    python-packages:
      - ocrmypdf
#      - cffi  # must be a setup and install requirement
#      - coloredlogs # strictly optional
#      - img2pdf  # pure Python, so track HEAD closely
#      - pdfminer.six
#      - pikepdf
#      - Pillow
#      - pluggy
#      - reportlab
#      - setuptools
#      - tqdm

    override-build: |
      snapcraftctl build
      sed -i 's/find_library(libname)/"liblept.so.5"/g' $SNAPCRAFT_PART_INSTALL/lib/python3.6/site-packages/ocrmypdf/leptonica.py

  cleanup:
    after:  # Make this part run last; list all your other parts here
      - ocrmypdfgui
    plugin: nil
    build-snaps:  # List all content-snaps and base snaps you're using here
      - core18
      - gnome-3-34-1804
      - gnome-3-34-1804-sdk
      - gtk-common-themes
    override-prime: |
      set -eux
      for snap in "core18" "gnome-3-34-1804" "gnome-3-34-1804-sdk" "gtk-common-themes"; do  # List all content-snaps and base snaps you're using here
          cd "/snap/$snap/current" && find . -type f,l -exec rm -f "$SNAPCRAFT_PRIME/{}" \;
      done

Error message on build:

Priming cleanup 
+ set -eux
+ cd /snap/core18/current
+ find . -type f,l -exec rm -f /root/prime/{} ;
+ cd /snap/gnome-3-34-1804/current
+ find . -type f,l -exec rm -f /root/prime/{} ;
+ cd /snap/gnome-3-34-1804-sdk/current
+ find . -type f,l -exec rm -f /root/prime/{} ;
+ cd /snap/gtk-common-themes/current
+ find . -type f,l -exec rm -f /root/prime/{} ;
Skipping prime gnome-3-34-extension (already ran)
The command 'python3' for 'python3 $SNAP/lib/python3.6/site-packages/ocrmypdfgui/__main__.py' was resolved to '/usr/bin/python3'.
The command 'python3 $SNAP/lib/python3.6/site-packages/ocrmypdfgui/__main__.py' has been changed to '/usr/bin/python3 $SNAP/lib/python3.6/site-packages/ocrmypdfgui/__main__.py'.
Failed to generate snap metadata: The specified command '/usr/bin/python3 $SNAP/lib/python3.6/site-packages/ocrmypdfgui/__main__.py' defined in the app 'ocrmypdfgui' does not match the pattern expected by snapd.
The command must consist only of alphanumeric characters, spaces, and the following special characters: / . _ # : $ -
Run the same command again with --debug to shell into the environment if you wish to introspect this failure.

What did I do wrong here?

13

Snapcraft doesn’t like the $ in your command. If everything is working correctly, your python modules should be in the python path and your command should work with just python3 -m ocrmypdfgui

14

unfortunately that does not fix it.

error message:

The command 'python3 -m ocrmypdfgui' has been changed to '/usr/bin/python3 -m ocrmypdfgui'.
Failed to generate snap metadata: The specified command '/usr/bin/python3 -m ocrmypdfgui' defined in the app 'ocrmypdfgui' does not match the pattern expected by snapd.
The command must consist only of alphanumeric characters, spaces, and the following special characters: / . _ # : $ -
Build failed

snapcraft.yaml:

name: ocrmypdfgui # you probably want to 'snapcraft register <name>'
title: ocrmypdfgui
base: core18 # the base snap is the execution environment for this snap
version: '0.9.05' # just for humans, typically '1.2+git' or '1.3.2'
summary: Hobby Project GUI for the Python Program "OCRmyPDF" by James R. Barlow
description: |
  I use James R. Barlow's OCRmyPDF heavily in my paperless Office and have created this Python Project as a GUI wrapper to run batch jobs on my filesystem. This is strictly a Hobby Project and is not "official". Feel free to use it if you like. Includes full current version of OCRmyPDF as backend. Icons and banner made by Freepik from www.flaticon.com
icon: gui/ocrmypdfgui.png
grade: devel # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots

architectures: [amd64]

environment:
    TESSDATA_PREFIX: $SNAP/usr/share/tesseract-ocr/4.00/tessdata
    GS_LIB: $SNAP/usr/share/ghostscript/9.26/Resource/Init
    GS_FONTPATH: $SNAP/usr/share/ghostscript/9.26/Resource/Font
    LD_LIBRARY_PATH: $SNAP/usr/lib/x86_64-linux-gnu

apps:
  ocrmypdfgui:
#    command: python3 $SNAP/lib/python3.6/site-packages/ocrmypdfgui/__main__.py #python3 $SNAP/lib/python3.6/site-packages/ocrmypdfgui/__main__.py
    command: python3 -m ocrmypdfgui
    desktop: $SNAPCRAFT_PROJECT_DIR/gui/ocrmypdfgui.desktop
    extensions: [gnome-3-34]

    plugs:
      - desktop
      - desktop-legacy
      - wayland
      - x11
      - home
      - removable-media


parts:
  ocrmypdfgui:
    # See 'snapcraft plugins'
    plugin: python
    source: .
    stage-packages: [ghostscript, icc-profiles-free, liblept5, libxml2, pngquant, tesseract-ocr-all, unpaper, qpdf, zlib1g]

    python-packages:
      - ocrmypdf
#      - cffi  # must be a setup and install requirement
#      - coloredlogs # strictly optional
#      - img2pdf  # pure Python, so track HEAD closely
#      - pdfminer.six
#      - pikepdf
#      - Pillow
#      - pluggy
#      - reportlab
#      - setuptools
#      - tqdm

    override-build: |
      snapcraftctl build
      sed -i 's/find_library(libname)/"liblept.so.5"/g' $SNAPCRAFT_PART_INSTALL/lib/python3.6/site-packages/ocrmypdf/leptonica.py

  cleanup:
    after:  # Make this part run last; list all your other parts here
      - ocrmypdfgui
    plugin: nil
    build-snaps:  # List all content-snaps and base snaps you're using here
      - core18
      - gnome-3-34-1804
      - gnome-3-34-1804-sdk
      - gtk-common-themes
    override-prime: |
      set -eux
      for snap in "core18" "gnome-3-34-1804" "gnome-3-34-1804-sdk" "gtk-common-themes"; do  # List all content-snaps and base snaps you're using here
          cd "/snap/$snap/current" && find . -type f,l -exec rm -f "$SNAPCRAFT_PRIME/{}" \;
      done
15

I think that’s a core18/python plugin thing… those restrictions don’t seem to exist on the latest python plugin for core20. Anyway, you may need to install a launcher script, and invoke that script instead of calling python directly.

16

Do I need to use desktop-launch here? The old version of my app used snapcraft-prelaunch to make tkinter work. Perhaps I need something akin to this…

17

Do you know when the Gnome3-38 Extensions for core20 will be ready? Right now the python installer seems to have problems - see above…

18

The gnome-3-38 extension for core20 has been stable for quite a while, I would recommend using that over gnome-3-34 if possible.

19

If it’s stable, do you know why python projects do not install to the path I expect them to? It works fine on core18 and gnome 3-34

20

The reason is because gnome-3-38 sets PYTHONPATH to its own python interpreter. But I’m sure you are asking why it does that… and I don’t know. It seems that the python plugin wasn’t considered in that decision.