Snap build started failing with pip not working

snapcraft
1

This past weekend it appears that fwupd snap builds have stopped working properly.

The failure shows up when trying to build meson.

Pulling meson 
Downloading 'meson-0.47.2.tar.gz'[] 100%

Failed to run '/root/project/parts/meson/install/usr/bin/python3 -m pip': Exited with code -11.

Manually running that command I get this:

/build/parts/meson/install/usr/bin/python3 -m pip
/build/parts/meson/install/usr/bin/python3: No module named pip

This is the part in snapcraft.yaml for meson.

  meson:
    plugin: python
    source: https://github.com/mesonbuild/meson/releases/download/0.47.2/meson-0.47.2.tar.gz
    build-packages:
      - ninja-build
    prime:
      - -bin
      - -etc
      - -lib
      - -share
      - -usr

It hasn’t changed for about 4 months: https://github.com/hughsie/fwupd/blob/master/snap/snapcraft.yaml

2

Something I should note is this seems like it was specifically occurring in the docker container snapcore/snapcraft based build that runs on CircleCI. I can reproduce that same failure locally in a docker container too.

However it seems to work fine on the Canonical Launchpad builder run at about the same time.
https://launchpadlibrarian.net/412720847/buildlog_snap_ubuntu_xenial_amd64_d2a8294152a080ac41aa11dcec60251a_BUILDING.txt.gz

Downloading 'meson-0.47.2.tar.gz'   0%                                         
[25/Feb/2019:09:50:10 +0000] "CONNECT github-production-release-asset-2e65be.s3.amazonaws.com:443 HTTP/1.0" 200 1276050 "-" "-"
[25/Feb/2019:09:50:10 +0000] "CONNECT github.com:443 HTTP/1.0" 200 6850 "-" "-"
Fetching and installing pip...
Collecting pip
[25/Feb/2019:09:50:11 +0000] "CONNECT pypi.python.org:443 HTTP/1.0" 200 4794 "-" "-"
  Downloading https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl (1.4MB)
  Saved ./parts/meson/python-packages/pip-19.0.3-py2.py3-none-any.whl
Successfully downloaded pip
[25/Feb/2019:09:50:12 +0000] "CONNECT files.pythonhosted.org:443 HTTP/1.0" 200 1362069 "-" "-"
[25/Feb/2019:09:50:12 +0000] "CONNECT pypi.org:443 HTTP/1.0" 200 15864 "-" "-"
Ignoring indexes: https://pypi.python.org/simple
Collecting pip
Installing collected packages: pip
Successfully installed pip-8.1.1
[25/Feb/2019:09:50:15 +0000] "CONNECT pypi.org:443 HTTP/1.0" 200 4569 "-" "-"
Fetching and installing wheel...
Collecting wheel
  Downloading https://files.pythonhosted.org/packages/96/ba/a4702cbb6a3a485239fbe9525443446203f00771af9ac000fa3ef2788201/wheel-0.33.1-py2.py3-none-any.whl
  Saved ./parts/meson/python-packages/wheel-0.33.1-py2.py3-none-any.whl
Successfully downloaded wheel
[25/Feb/2019:09:50:15 +0000] "CONNECT pypi.org:443 HTTP/1.0" 200 13124 "-" "-"
[25/Feb/2019:09:50:15 +0000] "CONNECT files.pythonhosted.org:443 HTTP/1.0" 200 27527 "-" "-"
Looking in links: /build/fwupd/parts/meson/python-packages
Collecting wheel
Installing collected packages: wheel
Successfully installed wheel-0.33.1
Fetching and installing setuptools...
Collecting setuptools
  Downloading https://files.pythonhosted.org/packages/d1/6a/4b2fcefd2ea0868810e92d519dacac1ddc64a2e53ba9e3422c3b62b378a6/setuptools-40.8.0-py2.py3-none-any.whl (575kB)
  Saved ./parts/meson/python-packages/setuptools-40.8.0-py2.py3-none-any.whl
Successfully downloaded setuptools
[25/Feb/2019:09:50:18 +0000] "CONNECT pypi.org:443 HTTP/1.0" 200 90429 "-" "-"
[25/Feb/2019:09:50:18 +0000] "CONNECT files.pythonhosted.org:443 HTTP/1.0" 200 582561 "-" "-"
Looking in links: /build/fwupd/parts/meson/python-packages
Collecting setuptools
Installing collected packages: setuptools
Successfully installed setuptools-40.8.0
Processing /build/fwupd/parts/meson/src
  Link is a directory, ignoring download_dir
Successfully downloaded meson
3

Hi, this is being tracked on LP: #1817300.
It only affects snaps using classic confinement, fix should be available everywhere in the coming days, this is where it currently works:

  • snapcraft from edge
  • launchpad/build.snapcraft.io

There was a bootstrapping issue as this affected snapcraft from building itself as well.
You can try and see if the fix works for docker by creating an image using edge.Dockerfile by following the documentation on https://github.com/snapcore/snapcraft/tree/master/docker

4

Here’s some form of explanation for the issue PSA: classic confinement and core snapcraft issue

1 Like
5

Thanks for explaining it. For now I’ve switched the CI for fwupd over to edge (https://github.com/hughsie/fwupd/commit/6121eb26df7938c833e988afa970bfcbe375fc16). When the fix is ready I’m guessing that LP bug will indicate so and I’ll switch back to stable.