I’m maintaining a python snap which was based on pip, using the python plugin but need to migrate that to uv. I see there’s also a corresponding plugin. Using that I get the following error:
Environment validation failed for part 'python-env': 'uv' not found and part 'python-env' does not depend on a part named 'uv-deps' that would satisfy the dependency.
The only somewhat-relevant reference I found concerns rust but that doesn’t do anything (beside no longer complaining that there’s no uv-deps). For completeness:
That should be all you need to migrate, but I’d need to see the full snapcraft.yaml to be sure. With this setup, I think you can omit the uv-deps part.
Awesome, thank you! That worked indeed. Did I miss that somewhere?
So I got it to compile and build the snap, but I’m not quite there yet. I want to use python3.11 but core22’s default is 3.10. I can’t quite get 3.11 to work, maybe you have an idea there as well?
Previously (pre-uv/with core20), I’d specify SNAPCRAFT_PYTHON_INTERPRETER: python3.11 in build-environment and also add python3.11 to stage-packages. Possibly also relevant: I use snapcraft-preload.
root@box:/# ls -alh $SNAP/bin
total 31K
drwxr-xr-x 2 root root 345 Oct 14 09:55 .
drwxr-xr-x 9 root root 188 Oct 14 09:55 ..
-rw-r--r-- 1 root root 4.1K Oct 14 09:54 activate
-rw-r--r-- 1 root root 2.7K Oct 14 09:54 activate.bat
-rw-r--r-- 1 root root 2.6K Oct 14 09:54 activate.csh
-rw-r--r-- 1 root root 4.2K Oct 14 09:54 activate.fish
-rw-r--r-- 1 root root 3.7K Oct 14 09:54 activate.nu
-rw-r--r-- 1 root root 2.8K Oct 14 09:54 activate.ps1
-rw-r--r-- 1 root root 2.4K Oct 14 09:54 activate_this.py
-rw-r--r-- 1 root root 1.7K Oct 14 09:54 deactivate.bat
-rwxr-xr-x 1 root root 324 Oct 14 09:54 f2py
-rwxr-xr-x 1 root root 341 Oct 14 09:54 normalizer
-rwxr-xr-x 1 root root 324 Oct 14 09:54 numpy-config
-rwxr-xr-x 1 root root 325 Oct 14 09:54 pinout
-rwxr-xr-x 1 root root 326 Oct 14 09:54 pintest
-rw-r--r-- 1 root root 1.2K Oct 14 09:54 pydoc.bat
lrwxrwxrwx 1 root root 49 Oct 14 09:54 python -> /root/parts/python-env/install/usr/bin/python3.11
lrwxrwxrwx 1 root root 6 Oct 14 09:54 python3 -> python
lrwxrwxrwx 1 root root 19 Oct 14 09:54 python3.11 -> /usr/bin/python3.10
-rwxr-xr-x 1 root root 119 Oct 14 09:55 snapcraft-preload
Note how python3.11 symlinks to /usr/bin/python3.10. I suspect that’s why I get a python3.10 shell where my packages aren’t available when I launch python3.11.
The symlink of python to the parts looks wrong as well, it’s of course not available. As far as I can tell, the problem is that my python version 3.11 doesn’t get packaged in the snap properly. How would I accomplish this?
I’m not totally sure how to interpret the part’s build output but there’s probably a clue in there:
Building python-env
:: + uv venv --relocatable --allow-existing --python /usr/bin/python3.10 /root/parts/python-env/install
:: Using CPython 3.10.12 interpreter at: /usr/bin/python3.10
:: warning: The requested interpreter resolved to Python 3.10.12, which is incompatible with the project's Python requirement: `==3.11.*` (from `project.requires-python`)
:: Creating virtual environment at: /root/parts/python-env/install
:: Activate with: source /root/parts/python-env/install/bin/activate
:: + PARTS_PYTHON_VENV_INTERP_PATH=/root/parts/python-env/install/bin/python3.11
:: + uv sync --no-dev --no-editable --reinstall
:: Using CPython 3.11.14 interpreter at: /root/parts/python-env/install/usr/bin/python3.11
:: Removed virtual environment at: /root/parts/python-env/install
:: Creating virtual environment at: /root/parts/python-env/install
...
Thanks, I saw that variable but I wasn’t sure what it meant or if/how it applied. Makes sense, I can build (when I include UV_PYTHON_DOWNLOADS: true) but I’m still missing the interpreter it seems.
I do need to bundle the interpreter as far as I understand, as it’s not part of the system.
The previous approach of stage-packages: [python3.11] doesn’t work and neither does stage-snaps: [astral-uv] (though it’s not clear to me what that would actually do - include the snap within my snap?).
I suppose that it’s not working is already visible in the end of the build output:
...
:: + set +e
:: + install_dir=/root/parts/python-env/install/usr/bin
:: + stage_dir=/root/stage/usr/bin
:: +++ readlink -f /root/parts/python-env/install/bin/python3
:: ++ basename /root/.local/share/uv/python/cpython-3.11.13-linux-aarch64-gnu/bin/python3.11
:: + basename=python3.11
:: + echo Looking for a Python interpreter called '"python3.11"' in the payload...
:: Looking for a Python interpreter called "python3.11" in the payload...
:: ++ find /root/parts/python-env/install/usr/bin /root/stage/usr/bin -type f -executable -name python3.11 -print -quit
:: ++ true
:: + payload_python=
:: + '[' -n '' ']'
:: + echo 'Python interpreter not found in payload.'
:: Python interpreter not found in payload.
:: + symlink_target=/usr/bin/python3.10
:: + '[' -z /usr/bin/python3.10 ']'
:: + eval 'set +o errexit'
:: ++ set +o errexit
:: + ln -sf /usr/bin/python3.10 /root/parts/python-env/install/bin/python3
… but it’s not clear to me what I need to do instead. (I get that both with stage-packages and stage-snaps.)
(On the upside, I see that uv now downloads and uses 3.11 )
As to $SNAP/build, I now have another (broken) symlink for python and python3 still points to python3.10 (which I think is related to the “interpreter not found in payload” during the build):
root@h-22010542:/# ls -alh $SNAP/bin
total 31K
drwxr-xr-x 2 root root 345 Oct 14 11:51 .
drwxr-xr-x 9 root root 188 Oct 14 11:51 ..
-rw-r--r-- 1 root root 4.1K Oct 14 11:51 activate
-rw-r--r-- 1 root root 2.7K Oct 14 11:51 activate.bat
-rw-r--r-- 1 root root 2.6K Oct 14 11:51 activate.csh
-rw-r--r-- 1 root root 4.2K Oct 14 11:51 activate.fish
-rw-r--r-- 1 root root 3.7K Oct 14 11:51 activate.nu
-rw-r--r-- 1 root root 2.8K Oct 14 11:51 activate.ps1
-rw-r--r-- 1 root root 2.4K Oct 14 11:51 activate_this.py
-rw-r--r-- 1 root root 1.7K Oct 14 11:51 deactivate.bat
-rwxr-xr-x 1 root root 324 Oct 14 11:51 f2py
-rwxr-xr-x 1 root root 341 Oct 14 11:51 normalizer
-rwxr-xr-x 1 root root 324 Oct 14 11:51 numpy-config
-rwxr-xr-x 1 root root 325 Oct 14 11:51 pinout
-rwxr-xr-x 1 root root 326 Oct 14 11:51 pintest
-rw-r--r-- 1 root root 1.2K Oct 14 11:51 pydoc.bat
lrwxrwxrwx 1 root root 77 Oct 14 11:51 python -> /root/.local/share/uv/python/cpython-3.11.13-linux-aarch64-gnu/bin/python3.11
lrwxrwxrwx 1 root root 19 Oct 14 11:51 python3 -> /usr/bin/python3.10
lrwxrwxrwx 1 root root 6 Oct 14 11:51 python3.11 -> python
-rwxr-xr-x 1 root root 119 Oct 14 10:10 snapcraft-preload
I also see my installed packages in $SNAP/lib/python3.11/site-packages/ so don’t think I’m that far off
Hmm, I did some more experiments and saw that even with the above, the interpreter was removed somehow during packing. I did manage to get a working 3.11 interpreter into a snap using uv like so:
Thanks so much for your effort! That worked for me as well, sort of
I get the symlink $SNAP/bin/python3 to $SNAP/usr/bin/python3.11 but I get an error when executing $SNAP/bin/python3 (more below). However, I can invoke python3 directly (or its target, $SNAP/usr/bin/python3.11) but then I can’t import my packages - it’s only checking $SNAP/usr/lib/... while my packages are in $SNAP/lib/....
I can import after modifying sys.path, but that’s not a good solution. I tried a few things to get these two consolidated, both with various uv flags/environment values and moving things around during the snap build, but wasn’t successful. What would you suggest here?
(That error I believe is why we added snapcraft-preload originally, at least it looks familiar:)
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = (not set)
program name = '/snap/agent/x25/bin/python3'
isolated = 0
environment = 1
user site = 1
safe_path = 0
import site = 1
is in build tree = 0
stdlib dir = '/install/lib/python3.11'
sys._base_executable = '/snap/agent/x25/usr/bin/python3.11'
sys.base_prefix = '/install'
sys.base_exec_prefix = '/install'
sys.platlibdir = 'lib'
sys.executable = '/snap/agent/x25/bin/python3'
sys.prefix = '/install'
sys.exec_prefix = '/install'
sys.path = [
'/install/lib/python311.zip',
'/install/lib/python3.11',
'/install/lib/python3.11/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x0000ffff9ea42020 (most recent call first):
<no Python frame>
A number of our plugins follow this pattern. It looks like Snapcraft accidentally drops this documentation for the uv plugin (I’ve opened an issue for this), but in the meantime, the upstream docs explain this more in-depth.
The Python plugins have a known issue that causes the Python interpreter to appear broken in override-* scripts. This is due to interactions between the expectations of Python virtual environments and our need to merge multiple file systems (from each part) into one.
This should be a bit easier in upcoming major releases (as any change would be breaking), but in the mean time, a common workaround is to explicitly declare your site-packages, as done here or here depending on the situation. The first should be done on a part that stages its own libraries, the second should be done on a part that just needs the libraries staged by a previous part.
If the issue persists after the snap has been packed, and @jnsgruk’s solution in this comment doesn’t work, I’m inclined to call this a new bug. But I would recommend trying the fixes you mentioned again, with the common workaround I linked above. It’s possible that one that didn’t work before will begin working with site-packages declared again.
Interesting, thanks for the insights! That part of the upstream docs would certainly have helped.
I’m not sure that second part applies to me though. I’m not (necessarily) looking to use uv directly in override-build (though yes, I noticed that it doesn’t work). The workaround with PYTHONPATH is a good hint though, I’m ok with just adding that to my app’s environment.
So that way my app can run my desired python version using my desired packages. That’s great! Now I’m at another roadblock though, some of the packages require additional libraries. Specifically, before I had:
I noticed that I no longer seem to need the build-packages and also the stage-packages don’t seem to make a difference - seems plausible that this is because I’m no longer using system python packages like before, but rather the astral-uv snap. Presumably that bundles additional libraries that are used at build time. So my question is: how do I bundle these libraries in the snap?
I get the following related linter warnings:
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libX11.so.6'. (provided by 'libx11-6') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libXau.so.6'. (provided by 'libxau6') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libXdmcp.so.6'. (provided by 'libxdmcp6') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libXext.so.6'. (provided by 'libxext6') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libXrender.so.1'. (provided by 'libxrender1') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libcairo.so.2'. (provided by 'libcairo2') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libfontconfig.so.1'. (provided by 'libfontconfig1') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libpixman-1.so.0'. (provided by 'libpixman-1-0') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libxcb-render.so.0'. (provided by 'libxcb-render0') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libxcb-shm.so.0'. (provided by 'libxcb-shm0') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/cairo/_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libxcb.so.1'. (provided by 'libxcb1') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi.cpython-311-aarch64-linux-gnu.so: missing dependency 'libgirepository-1.0.so.1'. (provided by 'libgirepository-1.0-1') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libX11.so.6'. (provided by 'libx11-6') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libXau.so.6'. (provided by 'libxau6') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libXdmcp.so.6'. (provided by 'libxdmcp6') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libXext.so.6'. (provided by 'libxext6') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libXrender.so.1'. (provided by 'libxrender1') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libcairo-gobject.so.2'. (provided by 'libcairo-gobject2') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libcairo.so.2'. (provided by 'libcairo2') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libfontconfig.so.1'. (provided by 'libfontconfig1') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libpixman-1.so.0'. (provided by 'libpixman-1-0') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libxcb-render.so.0'. (provided by 'libxcb-render0') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libxcb-shm.so.0'. (provided by 'libxcb-shm0') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
- library: lib/python3.11/site-packages/gi/_gi_cairo.cpython-311-aarch64-linux-gnu.so: missing dependency 'libxcb.so.1'. (provided by 'libxcb1') (https://documentation.ubuntu.com/snapcraft/stable/how-to/debugging/use-the-library-linter)
You would need to stage those using stage-packages, I would imagine. You just need to work out which packages provide those libraries and stage them in the snap (unless I’m missing something)
+1 to Jon’s response - the astral-uv snap shouldn’t change your final artifact since it’s just a build snap, as opposed to a stage snap, so anything it came with won’t be present at the end.
I tried that, unfortunately. I’m sorry, I wasn’t clear before - I had these build/stage packages before [and also tried with them now]. Since it was reporting libcairo/libgirepository as missing (both the linter and then python) I added that also to stage-packages but it didn’t make a difference.
I have an old version of the snap (built on core20, without uv), where I see gi.cpython-311-aarch64-linux-gnu.so and _gi_cairo.cpython-311-aarch64-linux-gnu.so in $SNAP/lib/python3.11/site-packages/gi. That’s without specifying them in stage-packages, I assume those are copied during the build step (hence the need in build-packages). For my updated snap it looks exactly the same.
What I’m unsure about is whether the problem is that the libraries are missing or just not being discovered. I don’t know how that kind of thing usually works - how would I determine that?
To be sure, were you using the exact name of the .so.2 file (e.g. libcairo.so.2) or the library it recommends (e.g. libcairo2)?
Situations can vary, but sometimes you may need something like libfoo-dev as a build-package but then libfoo as a stage-package.
As a more concrete example, including libcairo2-dev as a build-package and libcairo2 as a stage-package should result in /usr/lib/x86_64-linux-gnu/libcairo.so.2 being present inside your snap - do you see this? Does Snapcraft still complain even with that file there?
Interesting regarding libfoo-dev/libfoo. I still get the warning about missing dependency and I also don’t find libcairo.so.2. It’s built for ARM (raspi), so I assume it would be in /usr/lib/arm-linux-gnueabihf.
Full package list I’m using (I removed the -dev from libcairo2 in stage-packages but there’s no librepository1.0 so I left libgirepository1.0-dev).
I’ve been away for a few days but I got it to work now: I needed to move the stage-packages from the python-env part to the app part - could you maybe help figure out why?
I’m confused for two reasons, first, because it was working before (with core20/pip) when staging the packages in the python-env part and second, because I don’t understand why it would matter in the first place - don’t the outputs from every part end up in the snap anyway?
I wonder whether it’s related to the uv plugin that the python-env part uses. To test this, I added a separate part that just stages the packages - the following worked as well, could it be that stage-packages are somehow ignored by the uv plugin?
I also have another question regarding the intended usage of the uv plugin. I understand I should specify source: . when using a local uv.lock file. But that means the part re-builds whenever any file in the current directory changes. I can’t specify a file as the source - am I supposed to move uv.lock to a directory or add to a tar, or is there some other intended way?
Can I see your full python-env part? If you’re using something like the organize or stage keys, you might’ve been overriding those directories being staged. There’s a number of other possible things that might accidentally exclude those files - it’s a reasonably common pattern to make a part like your libs one to avoid this. Parts are a really nice system for this.
The uv plugin is meant to build a Python source project that uses uv to manage its dependencies. Consider a project with a foo/ directory for the tool’s Python source, a pyproject.toml describing a foo package, and a uv.lock file locking the dependencies described in pyproject.toml. Naturally, a source code change in foo/ would warrant a rebuild. In fact, any changes to a file in the directory specified by the source key will warrant a rebuild. What is your use-case where source code changes don’t deserve a rebuild?
Can I see your full python-env part? If you’re using something like the organize or stage keys, you might’ve been overriding those directories being staged. There’s a number of other possible things that might accidentally exclude those files - it’s a reasonably common pattern to make a part like your libs one to avoid this. Parts are a really nice system for this.
python-deps is the directory where I copy uv.lock, pyproject.toml, and my own package dependencies to (which are included as a git submodule and included as editable installs).
(I had to remove the python-related symlinks created by the plugin.)
The uv plugin is meant to build a Python source project that uses uv to manage its dependencies. Consider a project with a foo/ directory for the tool’s Python source, a pyproject.toml describing a foo package, and a uv.lock file locking the dependencies described in pyproject.toml. Naturally, a source code change in foo/ would warrant a rebuild. In fact, any changes to a file in the directory specified by the source key will warrant a rebuild. What is your use-case where source code changes don’t deserve a rebuild?
My pyproject.toml describes the python project but that itself isn’t installed as a package - I copy the snap’s “own” source code and execute that. Changing code in foo/ doesn’t require a uv re-install.
I use a separate step that copies the source code, that one I want to re-build on any change - but not the deps.
Hmm… Honestly, I’m not sure what’s going on here. I have a suspicion that override-prime may be acting up, but I’m not certain. Could you try replacing that override with this?
prime:
- -bin/python*
Why did you decide to separate the two? The uv plugin caches the builds just like it would in regular Snapcraft-free usage, so if the concern is performance then you shouldn’t need to do it. The plugin was designed with that performance in mind.
Hmm… Honestly, I’m not sure what’s going on here. I have a suspicion that override-prime may be acting up, but I’m not certain. Could you try replacing that override with this?
Ah yes, -bin/python* makes more sense. That works as well
My main issue there was that stage-packages don’t seem to get included in the part using the uv pugin, though.
Why did you decide to separate the two? The uv plugin caches the builds just like it would in regular Snapcraft-free usage, so if the concern is performance then you shouldn’t need to do it. The plugin was designed with that performance in mind.
Main reason for this is “because that’s how it was before” (with pip). Nice if this isn’t needed anymore, though I see that’s is removing and re-creating the virtual env every time - including building local packages and installing those as well as 3rd-party packages. Maybe I misunderstand what you’re expecting to be cached, though.
)
