Trouble bundling python with classic confinement in core20/4.0.4

Hello, I’m a developer on Certbot, EFF’s Let’s Encrypt plugin. We’re looking to migrate from the core18 base to the core20 base, and encountering some difficulties.

Basically, we have one snap that contains multiple python projects, some of which rely on each other. Since it’s just a venv now, I specify all of those packages in python-packages as ./package-name. It complains about being unable to build wheels, but I don’t think that’s actually a problem.

Instead, the problem seems to be that it’s not bundling python, despite doing as the docs say and listing a python3- package in stage-packages (since we use classic confinement). The shebang in the executable lists /usr/bin/env python3, which seems to mean that it doesn’t run correctly on other machines. Is there something else we should be doing as well to make sure python3 is bundled, and that we’re using the bundled python version?

Branch where I’ve been working on this is here; the commit log is fairly descriptive, but it might also help to see what we’re working with overall.

Hello @erica. I apologize as the python + core20 + classic combination still needs some further refinement. This is a known issue and we intend to improve the experience in the near future. When using core20, snapcraft reduces snap sizes by not staging anything found in the base snap, unless explicitly listed in the stage-packages. This unfortunately cause issues with python3 classic-mode snaps as you have seen.

I believe you should be OK as long as you stage python3 packages directly into your snap. Here is a test example we have for doing this: https://github.com/snapcore/snapcraft/blob/master/tests/spread/plugins/v2/snaps/python-hello-staged-python/snap/snapcraft.yaml

If you copy that list of stage-packages, I expect you’ll be in good shape. :slight_smile:

3 Likes

That seems to have done the trick, thank you!!!

2 Likes

Hello, I’m back here in 2024, updating for the core24 app. I’ve changed the snapcraft.yaml syntax to the newer versions and updated the list of packages to stage, and now we build perfectly well, and run on the machine it was built on, but don’t run when moving the snap to another (earlier) ubuntu machine. This makes me think that the packages are once again not getting staged/linked/patched properly, along with the snapcraft linter saying that libpython3.12.so.1.0 is unused.

The error I get when running on another machine is Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding, which seems to usually be a PYTHONPATH/PYTHONHOME issue, but that doesn’t quite make sense here. Looking at the variables the error spits out, on a working (older) version, when running /snap/certbot/current/bin/python3, sys.prefix is /snap/certbot/current, but on the updated version it’s now /usr; same with exec_prefix. And sys.path is similarly pointing to the /usr/lib pythons instead of inside the snap as well. So maybe this is a problem with patching somehow? I have now added build-attributes: enable-patchelf to the snapcraft.yaml, but did the actual patching get changed instead of just defaulting to no patching? Anything else I’m missing here? I’ve scoured the documentation but have run out of things to try.

All changes can be seen in this PR: Update python version in snaps by ohemorange · Pull Request #9956 · certbot/certbot · GitHub

Edit: this looks related snapcraft `8.0.2`+ python plugin classic core22 snap ModuleNotFoundError · Issue #4615 · canonical/snapcraft · GitHub

Edit 2: since I can’t change the topic title, I’m making a new one.