Package manager python dependencies not found in snap

Hi all!

I’m really hoping that someone will be able to point me in the right direction as I’m certain that I’m doing something wrong. Here’s my problem:

The snap I’m working on is called auto-cpufreq. It is essentially a python program that automates the CPU governor settings, turbo mode, etc., based on several factors. This python program depends on some libraries to be present on the path, which are happily installed with pip using the snapcraft python plugin.

Here is where things go off the rails: I’ve implemented some logging using the systemd python bindings (which allows for logging to the systemd journal), but it seems as though there have been longstanding issues with the systemd library in pip. Simply put, it barfs when you try to install it. However, all Linux distros have a python package that provides the same package, for instance, python3-systemd on Debian/Ubuntu or python-systemd on Arch. I install this by adding it to build-packages in the snapcraft.yaml, and all is well. Until…

After installing my locally built snap, I try to run it. The script breaks because it can’t find the python3-systemd that the package manager installed:

➜  auto-cpufreq git:(master) ✗ auto-cpufreq --stats
Traceback (most recent call last):
  File "/snap/auto-cpufreq/x1/bin/auto-cpufreq", line 14, in <module>
    from auto_cpufreq.core import *
  File "/snap/auto-cpufreq/x1/lib/python3.8/site-packages/auto_cpufreq/core.py", line 20, in <module>
    from systemd.journal import JournalHandler
ModuleNotFoundError: No module named 'systemd'

Is there an alternative way of making python aware of my installed dependencies in the snap? I tried adding the following to my snapcraft.yaml, but this doesn’t seem to make a difference:

    environment:
      PYTHONPATH: $SNAP/usr/lib/python3/dist-packages:$PYTHONPATH

I hope someone can point out where I’m going wrong.
Thanks in advance!

Strict confined snaps run within a private “mount namespace” that uses the chosen base snap as it’s root file system. So you won’t be able to see the host system’s Python installation or any modules installed for it. So if you need any Python modules outside of the standard library, you’ll need to include them in your snap.

With that said, even with a copy of python-systemd in your snap, I suspect it’ll be of limited use: the snap sandbox will block you from talking to systemd directly.

build-packages are used for build-time dependencies. If you want it included in the snap you’d want to add python3-systemd to stage-packages. Then I suspect your PYTHONPATH will work, but I don’t think it is required for this case.

Thanks @cjp256, I didn’t mention it in my post, but I already tried this without any effect. Incidentally, when I shell into the snap when running in devmode, I actually see systemd in the $SNAP/usr/lib/python3/dist-packages folder.

Thanks for the response @jamesh. So are you saying that adding this dependency to the dist-packages won’t include it into the snap? Maybe I’ve misunderstood the intention of this directive. Incidentally, when I shell into my snap, I definitely do see it on the file system.

If you push your snap to a branch I’ll be happy to take a look at it.

Thanks @cjp256, that would be greatly appreciated! You can find my fork the project here.

Ah, you got bit by a yaml technicality.

https://github.com/marc0der/auto-cpufreq/blob/master/snap/snapcraft.yaml#L47

You have a second environment: block overriding the first with PYTHONPATH. You just need to merge the two :smiley:

Awesome, thanks for your help, @cjp256. That did the trick! Sometimes it just takes a second pair of eyes to spot such things :slight_smile:

1 Like