Snapcraft container builds triggering a bug in snapd

[daniel]

trying to run snapcraft container builds:

Copy snap "core" data (cannot copy "/var/snap/core/2774" to "/var/snap/core/2774": failed to copy all: "cp: cannot stat '/var/snap/core/2774': No such file or directory" (1))

full log: http://paste.ubuntu.com/25472718/

@chipaca says it’s triggering a known snapd bug which has been fixed in master, but possibly snapcraft is doing something it shouldn’t.

I’m starting snapcraft with SNAPCRAFT_CONTAINER_BUILDS=1 snapcraft. lxd is configured and working fine via lxd init --auto I have had snapcraft build successfully previously, with exactly the same setup on this machine, but for some reason it has decided to stop working.

snap version on my host:

snap    2.27.5
snapd   2.27.5
series  16
ubuntu  17.04
kernel  4.10.0-33-generic

and in the container:

snap    2.27.5
snapd   2.27.5
series  16
ubuntu  16.04
kernel  4.10.0-33-generic

snapcraft --version on the host:

snapcraft, version 2.33+git53.69dceb8

[chipaca]

So: why is snapcraft refreshing from a version of core to the same version of core? It’ll be a NOP. Don’t do that!

(it shouldn’t fail with a nasty error, and that’s been fixed, but it’s still a waste of time)

[daniel]

it appears to be brute-forcing the snapd installation from an offline .snap file:

subprocess.CalledProcessError: Command '['lxc', 'exec', 'local:snapcraft-mars', '--', 'snap', 'install', '/run/core_2774.snap']' returned non-zero exit status 1.

So it’s failing having downloaded core_2774.snap directly rather than via refresh or install and then doing an installation via snap install /run/core_2774.snap from the now offline file.

If it doesn’t clean up the file when it fails in the manner above, then the file hasn’t actually been downloaded because it is currently non-existent at that path. (probably a misnomer, actually, because it shuts the container down after it runs and /run is a tmpfs)

[kalikiana]

@chipaca says it’s triggering a known snapd bug which has been fixed in master, but possibly snapcraft is doing something it shouldn’t.

Indeed it’s been discussed and fixed in snapd.

So: why is snapcraft refreshing from a version of core to the same version of core? It’ll be a NOP. Don’t do that!

For snapcraft to check beforehand if the version is the same is more error-prone than leaving that to snapd which does it anyway. Worst case a smallish file is copied and ignored.

[daniel]

This is reliably reproducible on my system. Simply:

1. remove any pre-existing lxd container from previous builds
2. run new build: SNAPCRAFT_CONTAINER_BUILDS=1 snapcraft
3. RE-run the command and notice that it worked the first time but failed the second

This is specific to re-using the same container multiple times, I believe. Something is being left in a wonky state the second time the container is booted.

[kalikiana]

Nothing wonky here. It’s literally the fact that the same snap is being snap installeded again.

[sergiusens]

This regardless of triggering a snapd bug, should be fixed on our side to not install if the version we want is already there (snaps with assertions will allow for proper checks).

[daniel]

it’s not just me hitting this: Snapcraft clean doesn’t clean with SNAPCRAFT_CONTAINER_BUILDS set

[kalikiana]

Fix is on its way https://github.com/snapcore/snapcraft/pull/1568