Building a snap with a multi Python project

snapcraft
1

Hello everyone !

So I am trying to build a snap for a project that contains multiple python projects, and fail to do so. The real use case can be found here: https://github.com/basak/certbot-snap-build. But I will simplify the case here to focus on the specific problem encountered.

Let’s suppose that I have a GIT repository, with two python project in it: project1 and project2. Each project is properly configured with setuptools, and so each of them has a setup.py. Also, project2 depends on project1. The two projects are on version 0.1.dev0 on the GIT master branch and are not released on PyPi. Since project2 requires new features added on project1 during development, its setup.py declares a dependency as follow: project1>=0.1.dev0.

If I want to setup a virtual environment with project2 and project1, I can run pip install -e project1 -e project2, and everything will work correctly since pip will see successfully that project1 on version 0.1.dev0 is installed locally when it tries to install project2.

However, I failed at building a snap in this configuration. A typical snapcraft.yml for this situation would leverage the python plugin into several parts like so:

parts:
  project1:
    plugin: python
    source: https://github.com/myrepo.git
    source-branch: master
    source-subdir: project1
  project2:
    plugin: python
    source: https://github.com/myrepo.git
    source-branch: master
    source-subdir: project2
    after: [project2]

In this case, snapcraft cannot build the snap. More precisely, the pull step on project2 fails with the typical following error:

ERROR: Could not find a version that satisfies the requirement project1>=0.1.dev0

It appears that snapcraft fails at seeing that project1 has been built and installed during the first part. So during the second part pip tries to retrieve project1 from PyPi on version 0.1.dev0, but this version has not been released.

Do you have some ideas to fix this situation, and allow the project2 part to reuse the modules from project1 part ?

Kindly regards,
Adrien Ferrand

2

This is a bug and we should fix.

3

Thanks a lot! Is there already a tracking issue, or also a schedule for fixing it?

If it is not assigned, I am willing to help and submit a PR.

4

Feel free to just submit the PR, my first thought to fix this is to look at the sitecustomize that is generated and include the python packages found in stage/lib into the path.

And thank you for taking interest in trying to work on this!

5

Sergio suggested to me this workaround:

parts:
    p:
        source: certbot.git
        plugin: python
        python-packages:
        - http://dependency.git

This did work when the snap was based on Xenial just now, but then I switched to base: core18 and the workaround no longer works and we are currently broken. I’m not clear on why. The log suggests that the python-packages directive worked, but subsequently even within the same part pip was unable to resolve the required dependency with the one previously pulled in.

The workaround also doesn’t look like it’ll work for certbot’s acme dependency, since acme is in a subdirectory of the certbot repository and as far as I can determine pip’s VCS URL syntax doesn’t support this case.

@sergiusens, @adferrand hasn’t been able to find the time to look into this more deeply. Please could you take a look? It’s causing the Certbot snap build in edge (built from master) to fail when upstream needs to make an API change - it’s red currently.

1 Like
6

For your information, the relevant Launchpad issue is here: https://bugs.launchpad.net/snapcraft/+bug/1841861