Cleanbuild fails differently locally than on snapcraft.io


#1

I’m trying to automate my snap but the cleanbuild fails on snapcraft.io. https://build.snapcraft.io/user/dmp1ce

I try to reproduce locally on my Vagrant setup but it fails in a different way. Do I need to initialize lxd in a certain way to get the same results as on snapcraft.io?


#2

Mind posting the build log or the error message?


#3

On snapcraft.io I think it is the errors with a proxy that many others are getting. Locally I am getting the following errors.

$ snapcraft cleanbuild --debug
Creating snapcraft-nonvendibly-imagerial-meaghan
Starting snapcraft-nonvendibly-imagerial-meaghan
error: flag provided but not defined: --format
Failed to get container image info: `lxc image list --format=json ubuntu:xenial/amd64` returned with exit code 2, output: b''
It will not be recorded in manifest.
Waiting for a network connection...
Network connection established
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [107 kB]
Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease
Get:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
Get:4 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [547 kB]
Get:5 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [232 kB]
Get:6 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
Get:7 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [365 kB]
Get:8 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [138 kB]
Get:9 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7532 kB]
Get:10 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [3456 B]
Get:11 http://security.ubuntu.com/ubuntu xenial-security/multiverse Translation-en [1744 B]
Get:12 http://archive.ubuntu.com/ubuntu xenial/universe Translation-en [4354 kB]
Get:13 http://archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [144 kB]
Get:14 http://archive.ubuntu.com/ubuntu xenial/multiverse Translation-en [106 kB]
Get:15 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [838 kB]
Get:16 http://archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [344 kB]
Get:17 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [678 kB]
Get:18 http://archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [274 kB]
Get:19 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [16.4 kB]
Get:20 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8344 B]
Get:21 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [6756 B]
Get:22 http://archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [4180 B]
Get:23 http://archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [7420 B]
Get:24 http://archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [3996 B]
Fetched 15.9 MB in 8s (1890 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  squashfuse
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 17.4 kB of archives.
After this operation, 54.3 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 squashfuse amd64 0.1.100-0ubuntu1~ubuntu16.04.1 [17.4 kB]
Fetched 17.4 kB in 0s (54.4 kB/s)
Selecting previously unselected package squashfuse.
(Reading database ... 25666 files and directories currently installed.)
Preparing to unpack .../squashfuse_0.1.100-0ubuntu1~ubuntu16.04.1_amd64.deb ...
Unpacking squashfuse (0.1.100-0ubuntu1~ubuntu16.04.1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up squashfuse (0.1.100-0ubuntu1~ubuntu16.04.1) ...
error: no changes of type "auto-refresh" found
Looking up assertion ['account-key', 'public-key-sha3-384=BWDEoaqyr25nF5SNCvEv2v7QnM9QsfCc0PBMYD_i2NGSQ32EF2d4D0hqUel3m8ul']
Looking up assertion ['snap-declaration', 'snap-name=core']
Looking up assertion ['snap-revision', 'snap-revision=5145', 'snap-id=99T7MUlRhtI3U0QFgl5mXXESAiSwt776']
error: open /var/lib/snapd/hostfs/home/vagrant/snap/lxd/common/snapcraftwov8mgug/core_5145.assert: no such file or directory
Stopping local:snapcraft-nonvendibly-imagerial-meaghan
Sorry, Snapcraft ran into an error when trying to running through its
lifecycle that generated the following traceback:
Traceback (most recent call last):
  File "/snap/snapcraft/1594/bin/snapcraft", line 11, in <module>
    load_entry_point('snapcraft==2.42.1', 'console_scripts', 'snapcraft')()
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/cli/lifecycle.py", line 222, in cleanbuild
    build_environment=build_environment)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_containers.py", line 53, in cleanbuild
    _deprecated_cleanbuild(project, remote, config, tar_filename)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_containers.py", line 77, in _deprecated_cleanbuild
    metadata=config.get_metadata(), remote=remote).execute()
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 186, in execute
    with self._container_running():
  File "/snap/snapcraft/1594/usr/lib/python3.5/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 84, in _container_running
    with self._ensure_started():
  File "/snap/snapcraft/1594/usr/lib/python3.5/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 107, in _ensure_started
    self._ensure_container()
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lxd/_cleanbuilder.py", line 50, in _ensure_container
    self._inject_snapcraft(new_container=True)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 253, in _inject_snapcraft
    self._inject_snap('core', tmp_dir)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 303, in _inject_snap
    ], tmp_dir)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 377, in _inject_assertions
    self._push_file(filepath, container_filename)
  File "/snap/snapcraft/1594/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 326, in _push_file
    src, '{}{}'.format(self._container_name, dst)])
  File "/snap/snapcraft/1594/usr/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['lxc', 'file', 'push', '/home/vagrant/snap/lxd/common/snapcraftwov8mgug/core_5145.assert', 'local:snapcraft-nonvendibly-imagerial-meaghan/run/core_5145.assert']' returned non-zero exit status 1


#4

Having the same issue.


#5

Here is more information on my issue:
Just looking for some help and/or perhaps confirmation of an issue:
snapcraft version: snapcraft, version 2.43
This is what I have done:

sudo snap install lxd
sudo lxd init
sudo snap install snapcraft --classic
sudo usermod -aG lxd ${USER}
newgrp lxd
snapcraft cleanbuild

This results on a permission issue. The trace for this is as follows:

    Traceback (most recent call last):
      File "/snap/snapcraft/1803/bin/snapcraft", line 11, in <module>
        load_entry_point('snapcraft==2.43', 'console_scripts', 'snapcraft')()
      File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 722, in __call__
        return self.main(*args, **kwargs)
      File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 697, in main
        rv = self.invoke(ctx)
      File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 895, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 535, in invoke
        return callback(*args, **kwargs)
      File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/cli/lifecycle.py", line 253, in cleanbuild
        build_environment=build_environment,
      File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_containers.py", line 57, in cleanbuild
        tar_filename = _create_tar_file(project.info.name)
      File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_containers.py", line 74, in _create_tar_file
        with tarfile.open(tar_filename, "w:bz2") as t:
      File "/snap/snapcraft/1803/usr/lib/python3.5/tarfile.py", line 1575, in open
        return func(name, filemode, fileobj, **kwargs)
      File "/snap/snapcraft/1803/usr/lib/python3.5/tarfile.py", line 1655, in bz2open
        compresslevel=compresslevel)
      File "/snap/snapcraft/1803/usr/lib/python3.5/bz2.py", line 95, in __init__
        self._fp = _builtin_open(filename, mode)
    PermissionError: [Errno 13] Permission denied: 'flameshot_source.tar.bz2'

If I run the command with sudo the build will succeed. If I uninstall the snap version of lxd and install the deb, I also get a permission issue. The issue with the deb version of lxd is identical to the one mentioned by the op.

The latest candidate doesn’t seem to make the situation better. In fact, it breaks cleanbuilds completely because even the sudo workaround becomes a no go.


#6

Try remove this file and retry, make sure every file under the source tree owns by you:

cd src-tree
sudo chown --recursive --changes $USER:$USER .

#8

Thanks got that a little better, but still errors:

Traceback (most recent call last):
  File "/snap/snapcraft/1803/bin/snapcraft", line 11, in <module>
    load_entry_point('snapcraft==2.43', 'console_scripts', 'snapcraft')()
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/cli/lifecycle.py", line 253, in cleanbuild
    build_environment=build_environment,
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_containers.py", line 60, in cleanbuild
    _deprecated_cleanbuild(project, remote, project_config, tar_filename)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_containers.py", line 88, in _deprecated_cleanbuild
    remote=remote,
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 241, in execute
    with self._container_running():
  File "/snap/snapcraft/1803/usr/lib/python3.5/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 90, in _container_running
    with self._ensure_started():
  File "/snap/snapcraft/1803/usr/lib/python3.5/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 112, in _ensure_started
    self._ensure_container()
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lxd/_cleanbuilder.py", line 56, in _ensure_container
    self._inject_snapcraft(new_container=True)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 316, in _inject_snapcraft
    self._inject_snap("snapcraft", tmp_dir)
  File "/snap/snapcraft/1803/lib/python3.5/site-packages/snapcraft/internal/lxd/_containerbuild.py", line 351, in _inject_snap
    shutil.copyfile(installed, filepath)
  File "/snap/snapcraft/1803/usr/lib/python3.5/shutil.py", line 114, in copyfile
    with open(src, 'rb') as fsrc:
PermissionError: [Errno 13] Permission denied: '/var/lib/snapd/snaps/snapcraft_1803.snap'

#9

This is more interesting, please post the following commands’ output from a terminal:

  • snapcraft version
  • lxd version

#10
lname:      lxd
summary:   System container manager and API
publisher: Canonical✓
contact:   https://github.com/lxc/lxd/issues
license:   unset
description: |
  LXD is a container manager for system containers.

  It offers a REST API to remotely manage containers over the network, using an image based workflow
  and with support for live migration.

  Images are available for all Ubuntu releases and architectures as well as for a wide number of
  other Linux distributions.

  LXD containers are lightweight, secure by default and a great alternative to virtual machines.
commands:
  - lxd.benchmark
  - lxd.buginfo
  - lxd.check-kernel
  - lxd.lxc
  - lxd
  - lxd.migrate
services:
  lxd.daemon: simple, enabled, active
snap-id:      J60k4JY0HppjwOjW8dZdYc8obXKxujRu
tracking:     stable
refresh-date: yesterday at 10:57 MDT
channels:
  stable:        3.4         (8415) 57MB -
  candidate:     3.4         (8424) 57MB -
  beta:          ↑
  edge:          git-aa490db (8540) 57MB -
  3.0/stable:    3.0.1       (8028) 57MB -
  3.0/candidate: 3.0.2       (8332) 56MB -
  3.0/beta:      ↑
  3.0/edge:      git-927a1b7 (8453) 56MB -
  2.0/stable:    2.0.11      (8023) 28MB -
  2.0/candidate: 2.0.11      (8023) 28MB -
  2.0/beta:      ↑
  2.0/edge:      git-92a4fdc (8000) 26MB -
installed:       3.4         (8415) 57MB -
name:      snapcraft
summary:   easily create snaps
publisher: Canonical✓
contact:   snaps@canonical.com
license:   unset
description: |
  Package, distribute, and update any app for Linux and IoT.

  Snaps are containerised software packages that are simple to create and install. They auto-update
  and are safe to run. And because they bundle their dependencies, they work on all major Linux
  systems without modification.
commands:
  - snapcraft
snap-id:      vMTKRaLjnOJQetI78HjntT37VuoyssFE
tracking:     stable
refresh-date: today at 16:06 MDT
channels:
  stable:    2.43                 (1803) 29MB classic
  candidate: 2.43.1               (1871) 29MB classic
  beta:      ↑
  edge:      2.43.1+git4.gbe4fb50 (1881) 32MB classic
installed:   2.43                 (1803) 29MB classic


#11

This is a known problem with a recent snapd version. It is installing snaps with read permission only provided for root, so your user account cannot access the snapcraft image to copy it into the lxd container. The solution is to execute a chmod command to make it readable:

sudo chmod +r /var/lib/snapd/snaps/snapcraft_*.snap