Snapcraft cannot cleanbuild in Snapped LXD

daniel 2017-07-24 15:23:25 UTC #1

I’ve previously used lxd natively, but thought I’d try in snapped lxd to really enter into the spirit of snap all the things. I installed lxd and snapcraft via snap. Snapcraft unfortunately seems unable to use the lxd delivered as a snap, as I received an error about being unable to find an lxd file or folder.

This is the process I followed along with error at the end:

$ sudo snap install --edge --classic snapcraft
$ sudo snap install lxd
$ sudo lxd init --auto
$ sudo snapcraft cleanbuild
Can not open /var/lib/snapd/seccomp/profiles//snap.lxd.lxc (No such file or directory)
aborting: No such file or directory
Something seems to be wrong with your installation of LXD.
Refer to the documentation at https://linuxcontainers.org/lxd/getting-started-cli.

sergiusens 2017-07-24 15:29:53 UTC #2

That message comes straight of the lxc client calling into lxd.

Can you tell me if this works (given which lxc produces a path a in /snap/bin/lxc):

lxc launch ubuntu:xenial

sergiusens 2017-07-24 15:30:58 UTC #3

Hm, this seems more like snap-confine trying to apply the seccomp filters. Maybe a better asked question to either @zyga or @tyhicks

daniel 2017-07-24 15:32:44 UTC #4

yes this works fine (when run through sudo):

$ sudo lxc launch ubuntu:xenial
Creating the container
Container name is: darling-monkey
Starting darling-monkey

sergiusens 2017-07-24 15:34:02 UTC #5

It should work without sudo, this is an indication of something being wrong. Can you do

sudo /snap/bin/lxc launch ubuntu:xenial
```?

daniel 2017-07-24 15:43:34 UTC #6

That works, too.

$ sudo /snap/bin/lxc launch ubuntu:xenial
Creating the container
Container name is: sunny-redbird
Starting sunny-redbird

sergiusens 2017-07-24 16:19:56 UTC #7

Ok, and without sudo?

daniel 2017-07-24 16:24:03 UTC #8

I get the standard permission denied about lxd group. There is no lxd group on the system, probably because snapd doesn’t create users or groups.

$ /snap/bin/lxc launch ubuntu:xenial
Permission denied, are you in the lxd group?

sergiusens 2017-07-24 16:31:47 UTC #9

Yeah, in the first message the error you got asked you to go to the lxd documentation, this step is needed for use with snapcraft: https://linuxcontainers.org/lxd/getting-started-cli/#access-control

daniel 2017-07-24 16:39:53 UTC #10

following those directions, I created a group called lxd and added myself to the group. I restarted snap.lxd.daemon.service. I ran newgrp lxd in a shell and then tried executing cleanbuild again. It fails with the same seccomp message as in the first instance without an lxd group:

Can not open /var/lib/snapd/seccomp/profiles//snap.lxd.lxc (No such file or directory)
aborting: No such file or directory
Something seems to be wrong with your installation of LXD.
Refer to the documentation at https://linuxcontainers.org/lxd/getting-started-cli.

sergiusens 2017-07-24 17:21:09 UTC #11

Which brings us back full circle to a seccomp issue and the two people I mentioned @zyga or @tyhicks, and let’s add @jdstrand just in case.

jdstrand 2017-07-24 17:36:12 UTC #12

FYI, I use the lxd snap with snapcraft all the time.

This is related to the seccomp bpf caching changes in 2.26. With 2.26, /var/lib/snapd/seccomp/profiles/snap.foo.bar doesn’t exist and we have /var/lib/snapd/seccomp/bpf/snap.foo.bar.src and /var/lib/snapd/seccomp/bpf/snap.foo.bar.bin instead. I suspect this has something to do with SNAP_REEXEC (and something @mvo would likely look at since he implemented the bpf caching feature).

Can you provide the output of snap version?

daniel 2017-07-24 17:38:13 UTC #13

$ snap version
snap    2.26.9
snapd   2.26.9
series  16
ubuntu  16.04
kernel  4.10.0-27-generic

mvo 2017-07-24 17:52:40 UTC #14

I tried the exact steps above but I get a different errror:

$ sudo snapcraft cleanbuild
"grade" property not specified: defaulting to "stable"
DEPRECATED: The 'snap' keyword has been replaced by 'prime'.
See http://snapcraft.io/docs/deprecation-notices/dn1 for more information.
LXD socket not found; is LXD installed and running?
There are either no permissions or the remote 'local' does not exist.
Verify the existing remotes by running `lxc remote list`
To setup a new remote, follow the instructions at
https://linuxcontainers.org/lxd/getting-started-cli/#multiple-hosts

@daniel is the error message a copy/paste? The “Can not open” is unusual, in our code we usually have “cannot” or “Cannot”.

daniel 2017-07-24 20:29:51 UTC #15

yes, it’s copy and pasted. here it is from a fresh invocation just to be sure that it is as I see it:

$ sudo snapcraft cleanbuild
"grade" property not specified: defaulting to "stable"
Can not open /var/lib/snapd/seccomp/profiles//snap.lxd.lxc (No such file or directory)
aborting: No such file or directory
Something seems to be wrong with your installation of LXD.
Refer to the documentation at https://linuxcontainers.org/lxd/getting-started-cli.

daniel 2017-07-26 14:30:22 UTC #16

is this maybe different under the HWE kernel (I’m running 16.04 xenial’s 4.10.0-27-generic HWE kernel)? There appears to be a regression that @jdstrand and I have been working to diagnose with this kernel and accessing /dev/zero (File_mmap of /dev/zero is currently blocked) so it surely isn’t inconceivable that there are other problems, such as maybe this one.

daniel 2017-07-26 15:07:31 UTC #17

ok, it seems somehow my system is in a mess - doing the same steps as above on a new install of 16.04.2 works fine - How do I reset my lxd installed by snap to virgin? I’ve tried removing it with snap remove lxd and reinstalling but it still doesn’t work, so there must be some remnants somewhere…

daniel 2017-07-31 10:42:21 UTC #18

I reinstalled my system. Something was really screwed with it causing this issue among others that started manifesting such as hanging shortly after bootup. Everything is now operating correctly and is stable. The only thing I’d done differently on the previous install was to accidentally set lxd to use zfs at one point.