Gnome extension conflicts with python plugin

lucyllewy · April 16, 2021, 3:18pm

Based on discussions with @sergiusens in the Snapcraft Clinic today we diagnosed an issue with the gnome extension (gnome-3-38 in this instance) breaking the python plugin by overriding the PATH and PYTHONPATH to point to the gnome extension paths which are missing python3-venv capability. (ref: Python plugin on core20 can't find ensurepip)

cc/ @kenvandine

James-Carroll · April 16, 2021, 5:33pm

Tangeantally related but I did notice that the gnome extensions (34 and 38) began defining pythonpath in the build time too (around January). This was a backwards incompatible change for me because the build system I was using was using Python2 and so erroring out with Py3 modules. Easily unset, but is one of the rare times I’d noticed breakage. Given I’m probably the only person mentioning 3 months later though I suppose it isn’t too problematic.

digidigital · September 19, 2021, 6:22pm

@sergiusens any news (fixes / proper documentation) regarding this topic?

I desperately try to snap a python application that uses the PySimpleGUI (tkinter) module with core20 / Gnome-3-38 and keep running in problems (The python plugin is working really nice with core18/ Gnome-3-34 ). I am not able to get the debs (python-tk) and modules in the snap (Path to Gnome-SDK-python / “Module xyz not found” when using gnome extension, missing plugs when and debs when not using the gnome extension etc.) and the documentation is outdated. Does anybody know of a working core20-python-tkinter-gnome3-38 snapcraft.yaml that I can use as a blueprint? Is it just my impression that almost all python-gui-apps are sticking with core18 (My program is bundled with an application that needs Glibc > 2.29 so I need core20)?

digidigital · September 26, 2021, 1:57pm

Since I was not able to get a working python with core20 / Gnome 3-38 I used pyinstaller as workaround to pack the used python version incl. all debs/modules and simply dumped the dist folder in the snap.

goodsoftworx · October 25, 2021, 2:02pm

I’m also stuck with this problem, a python app that uses gtk3. Seeing that the python plugin is not compatible with the gnome-3-38 extension in core20, I think I’ll have to change my code, which uses python features up to v3.9, to work with python 3.6 and core18. I tried upgrading python in core18 following instructions on this board, but ran into several problems with mixed python versions. I’m sure these conflicts will get sorted out one day, so maybe I should just be patient and focus on other things.

Edit: I gave it another go and got it working (core20, python3.9, gnome-3-38). It’s kind of brittle, because it manipulates PATH and PYTHONPATH to prevent a conflict with gnome-3-38’s python. Here’s the partial snapcraft.yaml in case this helps anyone:

base: core20

parts:
  myapp:
    plugin: python
    source: .

    build-environment:
      - SNAPCRAFT_PYTHON_INTERPRETER: python3.9
        # python3.9 lives in $SNAPCRAFT_PART_INSTALL/bin
      - PATH: $SNAPCRAFT_PART_INSTALL/bin:$PATH
      - PYTHONPATH: ''

    build-packages:
      - python3.9-venv
      - python3.9-dev
      # ...

    override-build: |
      # Work around a bug in snapcraft python plugin
      # https://forum.snapcraft.io/t/build-a-snap-with-any-version-of-python-i-want/10420/8
      rm -rf $SNAPCRAFT_PART_INSTALL/usr/lib/python3.9/distutils
      cp -r /usr/lib/python3.9/distutils $SNAPCRAFT_PART_INSTALL/usr/lib/python3.9/distutils
      mkdir -p $SNAPCRAFT_PART_INSTALL/usr/include/
      cp -r /usr/include/python3.9 $SNAPCRAFT_PART_INSTALL/usr/include/python3.9
      snapcraftctl build

    stage-packages:
      - python3.9-venv

apps:
  myapp:
    plugs:
      - mpris
      - audio-playback
      # ...

    extensions: [gnome-3-38]
    # must use the interpreter from bin/ (not usr/bin/)
    command: bin/python3.9 -m myapp

ppd · November 23, 2021, 2:53pm

Do we have a bug filed for this? I can’t find any on Launchpad or Gitlab. I reckon either the extension needs to not set PYTHONPATH, or the platform snap needs to include venv.

kenvandine · November 23, 2021, 5:17pm

Some improvements with python handling has landed in gnome-3-38-2004 this week, can you please try this again with that?

ppd · November 23, 2021, 5:31pm

with

name: py-core20-reproducer
base: core20
version: "1"
summary: py-core20-reproducer
description: py-core20-reproducer

grade: devel
confinement: devmode

apps:
  ls:
    command: /usr/bin/ls
    extensions:
      - gnome-3-38


parts:
  my-part:
    plugin: python
    python-packages:
      - camelcase

I get

Building my-part 
+ snapcraftctl build
+ python3 -m venv /root/parts/my-part/install
+ SNAPCRAFT_PYTHON_VENV_INTERP_PATH=/root/parts/my-part/install/bin/python3
+ pip install -U camelcase
Traceback (most recent call last):
  File "/snap/gnome-3-38-2004-sdk/current/usr/bin/pip", line 6, in <module>
    from pkg_resources import load_entry_point
ModuleNotFoundError: No module named 'pkg_resources'

James-Carroll · November 23, 2021, 5:36pm

Is there a chance there’s some misunderstanding here (and maybe that’s me!)

There’s two aspects in this thread from what I’m seeing. One is that the extensions write their desktop-launch wrapper with a PYTHONPATH set to use the Gnome content snaps. The other is that the during build time, having the Gnome extension defined at all for any singular app forces the PYTHONPATH variable on every single part in the snapcraft.yaml, regardless of whether the part makes use of GTK or Python3 (e.g, it’s breaking Python 2 for me during the buildtime of an Electron app for example).

They’re closely related but distinct IMO, in that it means the presence of the Gnome extensions at build time is breaking certain usages of Python that might be completely obscure to the maintainer. I’m less familiar with the runtime side of things because for me personally, I haven’t seen any regression (but I don’t do much with Python in snaps)

Infact, re-reading the title, it’s not even specifically the python plugin that’s impacted. It’s any plugin that just so happens to use Python (which the python plugin is probably the most common example of )

goodsoftworx · November 23, 2021, 9:25pm

I wonder if this extension was developed with an assumption that Python would be used for tools and helper/auxiliary scripts, rather than the main language for a desktop app.

hsbasu · September 25, 2022, 4:54am

I am having the same issue while snapping my app. I have prepared the .yml here. This builds the .snap file without any issue. But when I install and run the snap file using the following commands

snap install ./simple-pwgen_1.0.9snap+git1.6125061-dirty_amd64.snap  --dangerous --devmode
snap run simple-pwgen

I get the error:

Traceback (most recent call last):
  File "/snap/simple-pwgen/x29/bin/simple-pwgen", line 5, in <module>
    from SimplePwgen.main import start_SPGCli
ModuleNotFoundError: No module named 'SimplePwgen'

I wonder where I’ve messed up . Any help is highly appreciated .