Warpinator snap not working. When trying to run the snap fails with error message, "ModuleNotFoundError: No module named 'gi'"

saivinob · September 29, 2020, 12:08pm

Hi, I just started out with snapcrafting. I was able to snap couple of simple[r] programs (croc and youtube-dl) but I’m stuck with mint-tools. I’m trying to snap Warpinator, Webapp-Manager and Mintstick. Each of them fail with,

Failed to generate snap metadata: Specified command 'bin/<app-name>' was not found.
Verify the command is correct and for a more deterministic outcome, specify the relative path to the command from the prime directory.

I looked around and found one reply by @kyleN that is probably the issue I’m having but I do not know how to fix it. He asked to check if /parts/app-name/install/bin has the file. In my case they are not in that directory (for all the three apps I tried to snap) but then he suggests,

If that does not exist then you need to take a look at the plugin type (cmake in this case), since each plugin has different native approaches for installing files, and add appropriate install rules.

I have no idea what to be done. I think mintstick is using plain install.sh, webapp-manager has makefile and warpinator has meson build system. You may please take a look at their github page.

Below are my snapcraft.yaml. Please take a look and suggest what am I doing wrong.

mintstick snapcraft.yaml

name: mintstick
base: core18
version: '1.4.4'
summary: write .img and .iso files to USB sticks
description: |
  A GUI to write .img or .iso files to a USB Key. It can also format them.

grade: devel
confinement: devmode

parts:
  mintstick:
    source: https://github.com/linuxmint/mintstick.git
    plugin: make
    plugin: python
    python-version: python3
    stage-packages:
    - coreutils
    - gir1.2-glib-2.0
    - gir1.2-gtk-3.0
    - gir1.2-polkit-1.0
    - ntfs-3g
    - parted
    - policykit-1
    - procps
    - python3
    - python3-gi
    - python3-parted
    - gir1.2-udisks-2.0
    - gir1.2-xapp-1.0
    - util-linux
    - exfat-utils

apps:
  mintstick:
    command: bin/mintstick

I get following error.

    Cloning into '/root/parts/mintstick/src'...
    remote: Enumerating objects: 64, done.
    remote: Counting objects: 100% (64/64), done.
    remote: Compressing objects: 100% (44/44), done.
    remote: Total 1360 (delta 29), reused 44 (delta 17), pack-reused 1296
    Receiving objects: 100% (1360/1360), 536.94 KiB | 681.00 KiB/s, done.
    Resolving deltas: 100% (789/789), done.
    Fetching and installing pip...
    Collecting pip
      Cache entry deserialization failed, entry ignored
      Using cached https://files.pythonhosted.org/packages/4e/5f/528232275f6509b1fff703c9280e58951a81abe24640905de621c9f81839/pip-20.2.3-py2.py3-none-any.whl
      Saved /root/parts/mintstick/python-packages/pip-20.2.3-py2.py3-none-any.whl
    Successfully downloaded pip
    Collecting pip
    Installing collected packages: pip
    Successfully installed pip-20.2.3
    Fetching and installing wheel...
    Collecting wheel
      Using cached wheel-0.35.1-py2.py3-none-any.whl (33 kB)
      Saved /root/parts/mintstick/python-packages/wheel-0.35.1-py2.py3-none-any.whl
    Successfully downloaded wheel
    Looking in links: /root/parts/mintstick/python-packages
    Processing /root/parts/mintstick/python-packages/wheel-0.35.1-py2.py3-none-any.whl
    Installing collected packages: wheel
    Successfully installed wheel-0.35.1
    Fetching and installing setuptools...
    Collecting setuptools
      Using cached setuptools-50.3.0-py3-none-any.whl (785 kB)
      Saved /root/parts/mintstick/python-packages/setuptools-50.3.0-py3-none-any.whl
    Successfully downloaded setuptools
    Looking in links: /root/parts/mintstick/python-packages
    Processing /root/parts/mintstick/python-packages/setuptools-50.3.0-py3-none-any.whl
    Installing collected packages: setuptools
    Successfully installed setuptools-50.3.0
    Building mintstick 
    Staging mintstick 
    Priming mintstick 
    Failed to generate snap metadata: Specified command 'bin/mintstick' was not found.
    Verify the command is correct and for a more deterministic outcome, specify the relative path to the command from the prime directory.

webapp-manager snapcraft.yaml

name: webapp-manager
base: core18
version: '1.0.4'
summary: Webapp-Manager
description: |
  Run websites as if they were apps.

grade: devel
confinement: devmode

parts:
  webapp-manager:
    source: .
    plugin: make
    plugin: python
    stage-packages:
    - python3
    - python3-gi
    - python3-configobj
    - python3-setproctitle
    - python3-tldextract
    - xapps-common
    #organize:
    #  snap/$SNAPCRAFT_PROJECT_NAME/current: /

apps:
  webapp-manager:
    command: bin/webapp-manager

warpinator snapcraft.yaml

name: warpinator
base: core18
version: '1.0.8'
summary: Share files across the LAN
description: |
  Share files across the LAN.

grade: devel
confinement: devmode

parts:
  warpinator:
    source: .
    plugin: meson 
    plugin: python
    python-version: python3
    python-packages:
    - grpcio-tools
    - grpcio
    build-packages:
    - meson
    - python3-protobuf
    - gobject-introspection
    stage-packages:
    - gir1.2-glib-2.0
    - gir1.2-gtk-3.0
    - gir1.2-xapp-1.0
    - python3
    - python3-gi
    - python3-setproctitle
    - python3-xapp
    - python3-zeroconf
    - python3-cryptography
    - python3-nacl
    - python3-packaging

apps:
  warpinator:
    command: bin/warpinator
    plugs: ['home','network','network-bind','removable-media']

Sorry, if it sounds like homework question. If I can get them to snap, I’m planning to maintain them.

ogra · September 29, 2020, 12:47pm

i dont think this can work, you are defining two plugins for one part (in the subsequent snapcraft.yaml’s you seem to do this as well …) …

snapcraft should really error out when validating the yaml of the file, that looks like a bug in the validation …

kyleN · September 29, 2020, 1:11pm

In addition to what @ogra wrote, and for background, note that the snapcraft build process first builds each part separately and then combines them into the stage/ directory, and then converts that to the prime/ directory, from which the snap file is created.

Each part has three generated sub directories:
parts/PART/src <- this is populated when by the PULL step for the part
parts/PART/build <- populated by BUILD step for the part
parts/PART/install <- also populated by the BUILD step for the part

The way the part’s files move from src to build to install depends on the plugin type and any configurations or OVERRIDEs you can put in your snapcraft.yaml.

saivinob · September 30, 2020, 9:03am

@ogra, thank you for this pointer. Thanks @kyleN.

Obviously I haven’t read the documents completely but jumped straight in after watching Alan Pope’s and Daniel Holbach’s videos

Since warpinator was progressing better than the other two, I concentrated on that. Now I’ve built the snap but when I try to launch it, I get gi module import error as below.

Traceback (most recent call last):
  File "/snap/warpinator/x10/usr/local/bin/warpinator", line 7, in <module>
from gi.repository import Gio, GLib
ModuleNotFoundError: No module named 'gi'

It is included in the snapcraft.yaml as a ‘stage-package’, though. Can you please take a look.

name: warpinator
base: core18
version: '1.0.8'
summary: Share files across the LAN
description: |
  Share files across the LAN.

grade: devel
confinement: devmode

parts:
  mint-grpc:
source: .
plugin: nil
override-build: |
  echo "deb http://ppa.launchpad.net/clementlefebvre/grpc/ubuntu bionic main" > /etc/apt/sources.list.d/grpc.list
  apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1175170511CC89B482DB6F74FBC9C71605F55EC4

  warpinator:
source: .
plugin: meson 
build-packages:
- appstream
- gcc
- gettext
- gnome-pkg-tools
- meson
- policykit-1
- python3
- python3-grpcio
- python3-grpc-tools
- python3-protobuf
- gobject-introspection
stage-packages:
- gir1.2-glib-2.0
- gir1.2-gtk-3.0
- gir1.2-xapp-1.0
- python3
- python3-gi
- python3-setproctitle
- python3-xapp
- python3-zeroconf
- python3-grpcio
- python3-protobuf
- python3-cryptography
- python3-nacl
- python3-netifaces
- python3-packaging

apps:
  warpinator:
command: bin/warpinator
#plugs: ['home','removable-media','network','network-bind']

Also, regarding ppa, I needed this for two dependencies, grpcio and grpc-tools which are not available in bionic. Is there a better way to handle this? Can we directly add a ppa, say as source? There is no apt-add-repository command.

saivinob · September 30, 2020, 5:27pm

I added extensions under apps as,

extensions: [gnome-3-34]

Nil joy. Snap builds successfully but still doesn’t launch with “no module name ‘gi’” error as mentioned above.

ogra · October 1, 2020, 9:25am

looks like you need to mangle PYTHONPATH to point to the right place …

alternatively you could pull all the python modules from pip witn a separate python part, that should set PYTHONPATH for you…

saivinob · October 1, 2020, 4:50pm

I had read your suggestion regarding this elsewhere in the forum and had tried adding PYTHONPATH but does not help. I have added following line.

apps:
  warpinator:
    extensions: [gnome-3-28]
    command: bin/warpinator
    environment:
      PYTHONPATH: $SNAP/usr/lib/python3.6/site-packages:$SNAP/usr/lib/python3/dist-packages:$PYTHONPATH

It goes past ‘gi’ but then errors saying no module named ‘warpinator’.

cjp256 · October 2, 2020, 12:54pm

I believe this was just fixed by @hellsworth and merged into snapcraft two days ago. https://github.com/snapcore/snapcraft/pull/3270/files

gi should come from the gnome-platform snap, which means you should be OK with “$SNAP_DESKTOP_RUNTIME/usr/lib/python3.6/site-packages” in your PYTHONPATH.

Or sudo snap refresh snapcraft --edge and see if it works for you

cjp256 · October 2, 2020, 12:57pm

@hellsworth The above snap is using 3-28, and for that case the path may need to be $SNAP_PLATFORM/usr/lib/python3/dist-packages/gi in that case.

@saivinob if trying with edge, using gnome-3-34

saivinob · October 3, 2020, 5:06am

@cjp256, thank you for the suggestion and the link. I tried to build with gnome-3-34, snap completes build as before but on launching shows segmentation fault. Please see complete message below.

sai@sai-mintstick:~/snapcrafts/warpinator$ warpinator 
Warning: Schema “org.gnome.system.locale” has path “/system/locale/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy” has path “/system/proxy/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy.http” has path “/system/proxy/http/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy.https” has path “/system/proxy/https/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy.ftp” has path “/system/proxy/ftp/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy.socks” has path “/system/proxy/socks/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Segmentation fault (core dumped)

I tried,

snapcraft stable + gnome-3-34, same error of module ‘gi’ not found
snapcraft stable + gnome-3-34 + environment set, seg faults
snapcraft edge + gnome-3-34, seg faults
snapcraft edge + gnome-3-34 + environment set, seg faults
did snapcraft clean and then rebuilt with, snapcraft edge + gnome-3-34, seg faults.

By environment set, I mean this,

environment:
      PYTHONPATH: $SNAP_DESKTOP_RUNTIME/usr/lib/python3.6/site-packages:$SNAP/lib/python3.6/site-packages:$SNAP/usr/lib/python3/dist-packages:$PYTHONPATH

and also,

environment:
      PYTHONPATH: $SNAP/lib/python3.6/site-packages:$SNAP/usr/lib/python3/dist-packages:$PYTHONPATH

When I check, /snap/warpinator/current/gnome-platform, it is empty.

sai@sai-mintstick:~/snapcrafts/warpinator$ ls -la /snap/warpinator/current/gnome-platform/
total 0
drwxr-xr-x  2 root root   3 Oct  3 09:58 .
drwxr-xr-x 11 root root 156 Oct  3 10:00 ..

Following is the snapcraft.yaml I am using.

name: warpinator
base: core18
version: '1.0.8'
summary: Share files across the LAN
description: |
  Share files across the LAN.

grade: devel
confinement: devmode

package-repositories:
  - type: apt
    ppa: clementlefebvre/grpc

parts:
  mint-grpc:
    plugin: nil
    build-packages:
    - software-properties-common
    - apt-utils
    - python3-grpcio
    - python3-grpc-tools
    - python3-protobuf
    #stage-packages:
    #- python3-grpcio
    #- python3-grpc-tools
    #- python3-protobuf
    
  pymodules:
    source: .
    plugin: python
    python-version: python3
    stage-packages:
    - python3-gi
    - python3-setproctitle
    - python3-xapp
    - python3-zeroconf
    - python3-cryptography
    - python3-nacl
    - python3-netifaces
    - python3-packaging
    - python3-grpc-tools
    - python3-grpcio
    - python3-protobuf
    after: [mint-grpc]
   
  warpinator:
    source: .
    plugin: meson 
    meson-parameters: [ --prefix=/usr ]
    build-packages:
    - appstream
    - gcc
    - gettext
    - gnome-pkg-tools
    - meson
    - policykit-1
    - gobject-introspection
    stage-packages:
    - gir1.2-glib-2.0
    - gir1.2-gtk-3.0
    - gir1.2-xapp-1.0
    after: [pymodules]

apps:
  warpinator:
    extensions: [gnome-3-34]
    command: bin/warpinator
    #environment:
    #  PYTHONPATH: $SNAP/lib/python3.6/site-packages:$SNAP/usr/lib/python3/dist-packages:$PYTHONPATH
    #  PYTHONPATH: $SNAP_DESKTOP_RUNTIME/usr/lib/python3.6/site-packages:$SNAP/lib/python3.6/site-packages:$SNAP/usr/lib/python3/dist-packages:$PYTHONPATH
    #plugs: ['home','removable-media','network','network-bind']

ogra · October 3, 2020, 10:12am

try adding:

layout:
  /usr/bin/python3.6:
    bind-file: $SNAP/usr/bin/python3.6

that should make the segfault go away, i guess the python code runs #!/usr/bin/python3 instead of #!/usr/bin/env python3 (which would pick the interpreter from inside the snap)… to make such code use the shipped interpreter instead of the hosts one…

saivinob · October 3, 2020, 5:11pm

Yes, the python code are all using #!/usr/bin/python3 as you guessed. However adding the layout has not changed the situation. Without environment set, it seg faults. With environment set, it just says no module ‘warpinator’ (goes past ‘gi’).

I think I saw python seem to be symlink to python2 instead of python3 in the base environment. I did not take screenshot. Is that normal, perhaps for Bionic? Does that affect? Also, is it normal that gnome-platform is an empty folder?

cjp256 · October 6, 2020, 2:49am

Yes. You can see it if you enter the context of your snap: snap run --shell warpinator

It looks like a mismatch between the glib libs staged in your snap vs. the ones in the gnome desktop.

$ snap run --shell warpinator
$ python3 -c 'from gi.repository import Gio, GLib'
Segmentation fault (core dumped)

$ LD_LIBRARY_PATH=$SNAP/gnome-platform/lib/x86_64-linux-gnu:$SNAP/gnome-platform/usr/lib/x86_64-linux-gnu:$SNAP/gnome-platform/usr/lib:$SNAP/gnome-platform/lib:$SNAP/gnome-platform/usr/lib/x86_64-linux-gnu/dri:$LD_LIBRARY_PATH

$ python3 -c 'from gi.repository import Gio, GLib'
...OK.

$ $SNAP/usr/bin/warpinator
No module named 'warpinator'

That last error is because there is no warpinator library found in the snap. Probably an easy fix once we sort out how to resolve the segfault…

LD_LIBRARY_PATH is modified late, so we can’t just set the order in the app’s environment. Is there an established pattern to address this @kenvandine @hellsworth @sergiusens ?

cjp256 · October 6, 2020, 2:26pm

Remove the following stage-packages which conflict with the gnome platform snap:

saivinob · October 7, 2020, 7:44am

If I do this , I am getting following error.

sai@sai-mintstick:~/snapcrafts/warpinator$ warpinator
/snap/warpinator/x3/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3-0/gtk-query-immodules-3.0: symbol lookup error: /snap/warpinator/x3/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3.so.0: undefined symbol: g_mount_operation_set_is_tcrypt_hidden_volume
ERROR: /snap/warpinator/x3/gnome-platform/usr/lib/x86_64-linux-gnu/libgtk-3-0/gtk-query-immodules-3.0 exited abnormally with status 127
Traceback (most recent call last):
  File "/snap/warpinator/x3/usr/bin/warpinator", line 7, in <module>
    from gi.repository import Gio, GLib
ModuleNotFoundError: No module named 'gi'

My latest snapcraft.yaml is as below:

name: warpinator
base: core18
version: '1.0.8'
summary: Share files across the LAN
description: |
  Share files across the LAN.

grade: devel
confinement: devmode
license: GPL-3.0+

package-repositories:
  - type: apt
    ppa: clementlefebvre/grpc

layout:
  /usr/bin/python3.6:
    bind-file: $SNAP/usr/bin/python3.6

parts:
  pymodules:
    source: .
    plugin: python
    python-version: python3
    build-packages:
    - python3-grpcio
    - python3-grpc-tools
    - python3-protobuf
    stage-packages:
    - python3-grpc-tools
    - python3-grpcio
    - python3-protobuf
    #- python3-gi
    - python3-setproctitle
    - python3-xapp
    - python3-zeroconf
    - python3-cryptography
    - python3-nacl
    - python3-netifaces
    - python3-packaging
   
  warpinator:
    source: .
    plugin: meson 
    meson-parameters: [ --prefix=/usr ]
    build-packages:
    - software-properties-common
    - apt-utils
    - appstream
    - gcc
    - gettext
    - gnome-pkg-tools
    - meson
    - policykit-1
    - gobject-introspection
    stage-packages:
    #- gir1.2-glib-2.0
    #- gir1.2-gtk-3.0
    #- gir1.2-xapp-1.0
    #- gir1.2-gobject-2.0
    #- gir1.2-cairo-1.0
    - gettext
    after: [pymodules]

icon: snap/gui/org.x.Warpinator.png

apps:
  warpinator:
    command: usr/bin/warpinator
    extensions: [gnome-3-34]
    #environment:
    #  PYTHONPATH: $SNAP/usr/lib/python3/dist-packages:$SNAP/usr/lib/$ARCH:$PYTHONPATH
    #plugs: ['home','removable-media','network','network-bind']

In past couple of days, I scanned snapstore for python+gtk snaps to see how they have written snapcraft.yaml (if it’s on snapstore, obviously it works!). I found two: bzoing and kfit. I tried to create snap for both locally and snapped successfully. They also run fine. They had used desktop-launch/desktop-helpers method. To see, if it is an issue with extensions method, I changed the snapcraft.yaml file to use extensions. Still both apps snapped and work correctly. I also created snap for another python+gtk app called ‘exaile’, this time creating snap from scratch. It also built and worked OK (except I did not have icons for any theme other than Ubuntu Mono/Humanity). Issue is only with warpinator (mint tools?!).

As much as I could see, all of the other projects used /usr/bin/env python3 while warpinator uses /usr/bin/python3 as @ogra had asked me before. But then, with same, I was able to create a .deb package and run without issue. I also installed flatpak version and that also works OK. So, the problem is on the snap-side.

cjp256 · October 7, 2020, 2:28pm

I reproduced your issue with the yaml you just posted.

Here is the yaml I tested yesterday, that got further.

name: warpinator
base: core18
version: '1.0.8'
summary: Share files across the LAN
description: |
  Share files across the LAN.

grade: devel
confinement: devmode

package-repositories:
  - type: apt
    ppa: clementlefebvre/grpc

parts:
  mint-grpc:
    plugin: nil
    build-packages:
    - software-properties-common
    - apt-utils
    - python3-grpcio
    - python3-grpc-tools
    - python3-protobuf
    
  pymodules:
    source: .
    plugin: python
    python-version: python3
    stage-packages:
    - python3-setproctitle
    - python3-xapp
    - python3-zeroconf
    - python3-cryptography
    - python3-nacl
    - python3-netifaces
    - python3-packaging
    - python3-grpc-tools
    - python3-grpcio
    - python3-protobuf
    after: [mint-grpc]
   
  warpinator:
    source: .
    plugin: meson 
    meson-parameters: [ --prefix=/usr ]
    build-packages:
    - appstream
    - gcc
    - gettext
    - gnome-pkg-tools
    - meson
    - policykit-1
    - gobject-introspection
    after: [pymodules]

apps:
  warpinator:
    extensions: [gnome-3-34]
    command: bin/warpinator

Staging gettext looks like the culprit in your case. You should also be able to drop the layout.

saivinob · October 7, 2020, 3:42pm

I tried as suggested. There are no seg faults or earlier error messges now but am back to plain ‘no module named ‘gi’’ error.

Traceback (most recent call last):
  File "/snap/warpinator/x3/usr/bin/warpinator", line 7, in <module>
    from gi.repository import Gio, GLib
ModuleNotFoundError: No module named 'gi'

cjp256 · October 7, 2020, 4:24pm

Can you try a snapcraft clean in case something got messed up? I don’t know why you would still be getting that result…

saivinob · October 7, 2020, 5:03pm

I did that. Actually, each time I am building now, I am starting with snapcraft clean just to be sure. Hasnt’ made any difference.

I tried to build using desktop-gtk3/desktop-lauch method (not using extension). Builds fine but same error for ‘gi’. I then ran the snap run --shell command you showed before. I get following. Hope it helps.

sai@sai-mintstick:~/snapcrafts/warpinator$ snap run --shell warpinator
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

sai@sai-mintstick:/home/sai/snapcrafts/warpinator$ python3 -c 'from gi.repository import Gio, Glib'
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 888, in _find_spec
AttributeError: 'DynamicImporter' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/snap/warpinator/x12/usr/lib/python3/dist-packages/gi/importer.py", line 127, in find_module
    'introspection typelib not found' % namespace)
ImportError: cannot import name Gio, introspection typelib not found

Did you mean, it worked for you? You could launch warpinator GUI?

cjp256 · October 7, 2020, 6:40pm

No, but I had No module named 'warpinator'. Which is because of how warpinator installs the python module.

Getting past that, I now get ValueError: Namespace XApp not available. Which brings us back around to the GTK mismatch because we need gir1.2-xapp-1.0 …

Here’s what I’m working from: https://github.com/cjp256/warpinator/blob/snap-support/snap/snapcraft.yaml

Hopefully @kenvandine @hellsworth can help us out… should we be using desktop helpers for this case?