Build on LXD

[wiki]

Snapcraft, the snap-building tool, is designed to use a combination of Multipass and bases to both simplify the build process and to confine the build environment within a virtual machine. This mostly removes the need to use LXD.

ⓘ See Snapcraft overview for details on both Multipass and bases

Snapcraft can also use LXD, and it’s often a faster alternative.

• If you are using bases and Snapcraft version 3.4+ you can request to use LXD with the --use-lxd argument:

  $ snapcraft --use-lxd
  

  You can also set SNAPCRAFT_BUILD_ENVIRONMENT=lxd to use LXD by default.

• If your snapcraft.yaml doesn’t use bases, you should continue using snapcraft cleanbuild.

See below for help on installing LXD, manually building snaps within an LXD environment and building without bases.

Install LXD

Recent non-desktop versions of Ubuntu install LXD by default - you can check whether it is installed with the following command:

$ lxd version
3.13

The easiest way to add LXD to your system is via its snap:

$ sudo snap install lxd

On recent versions of Ubuntu, the deb package encapsulates the snap installation.

Now initialise LXD with the following command, accepting all the default options unless you have specific requirements:

$ sudo lxd init

ⓘ If the system you are installing LXD onto is using a network with a 10.x.x.x subnet then network create may fail. Step through the following to resolve this.

Manually create a network on a 10.x.x.x subnet

If you try to run lxd init on a system that is connected to a network with a 10.x.x.x subnet, then the final step of the init* may fail with the following error:

Error: Failed to create network 'lxdbr0': Failed to automatically find an unused IPv4 subnet, manual configuration required

To resolve this issue, you must manually create the network with the following command:

$ sudo lxc network create lxdbr0 ipv4.address=10.0.3.1/24 ipv4.nat=true

You can then re-run lxd init. When you are prompted to create a new network bridge you must respond no.

Would you like to create a new network bridge (yes/no) [default=yes]? no

Group permissions

If you want to build snaps as a non-root user, which is advised, then you need to add your user account to the lxd group:

$ sudo usermod -a -G lxd ${USER}

You now need to either restart your session, reboot your computer, or use newgrp to acquire the new group assignment:

$ newgrp lxd

ⓘ The newgrp command will start a new sub-shell (shell within a shell) with the new lxd group assigned.

Building with bases

Snapcraft with LXD operates in much the same way as snapcraft with a default Multipass environment. However, with LXD installed and configured, you need to manually add the --use-lxd argument to initiate the build within LXD, or set the SNAPCRAFT_BUILD_ENVIRONMENT=lxd environment variable.

$ snapcraft --use-lxd

Many of the snapcraft’s Multipass operations also work with LXD, including:

• --shell
• --shell-after
• --debug

Similarly, snapcraft’s lifecycle commands, pull, build, stage and prime, together with clean, can also use the --use-lxd option.

$ snapcraft clean --use-lxd
The LXD provider is offered as a technology preview for early adopters.
The command line interface, container names or lifecycle handling may change in upcoming releases.

Building without bases

ⓘ If snapcraft is not yet installed, enter sudo snap install snapcraft --classic.

To build your project using lxd run:

$ cd $HOME/my-snap-project
$ snapcraft cleanbuild

If the build fails, add the --debug option to the snapcraft command. This will drop you into a shell running within the LXD environment that just attempted to build your snap, allowing you to examine where the issue may be.

Building manually

These instructions are intended to be only a general guide. For further details on using LXD as a container environment, see the LXD Documentation.

First, create and run a new container based on Ubuntu 18.04 LTS (Bionic Beaver). Our example calls this container mysnapcraft:

$ lxc launch ubuntu:18.04 mysnapcraft

Copy your snap’s snapcraft.yaml to this new container:

$ lxc file push snap/snapcraft.yaml mysnapcraft/home/ubuntu/

Now open an interactive shell within your container and install snapcraft:

$ lxc exec mysnapcraft -- /bin/bash 
$ snap install snapcraft --classic

Finally, staying within the container, start the build by running snapcraft with the --destructive-mode argument. This forces snapcraft to build the snap directly within the current host (the mysnapcraft LXD container):

$ cd /home/ubuntu
$ snapcraft --destructive-mode

You can troubleshoot the build process just as you would on the native machine. The container is persistent and will remain until stopped and deleted.

With the build complete, you can copy your new snap to your native environment with the following command:

$ lxc file pull mysnapcraft/home/ubuntu/mysnap.snap .

[popey]

It looks like you’ve copy/pasted the docs directly from docs.snapcraft.io to here. I’d advise against that. We’re currently in the process of re-writing some of the docs there and copy/pasting old copies around the place will leave stale docs here.

The docs you’ve copied over are also needlessly over-complicated (which is why I’m mid way through re-writing them) and will confuse new users. So I’d recommend we simplify them in one place for now.

[bsutton]

It wasn’t a straight copy paste as the old version was out of date.

I’ve done a re-write (based on my understanding) and I’m in the process of testing the steps.

If there are simpler steps then let me know.

The docker instructions were a copy/paste.

Were are you re-writing the docs?
I thought the process was via the forums?

And just some feedback.

One of the key problems with the current docs is a lack of details.
I agree we need a broad overview but we really need to allow users to drill down into more detail.
There is far too much that is left unsaid in the name of keeping things simple.
I haven’t found the current docs very useful specifically because of this problem.

[popey]

The docs are hosted at docs.snapcraft.io - the source for them is on github. We welcome changes to the docs. I’m mid way through re-writing some of the pages and it would be great if we could work together on those docs rather than have 2 (now 3) places for them. While Gustavo has a documentation experiment website up, the official docs have been and still are at docs.snapcraft.io.

[bsutton]

@popey

rather than have 2 (now 3) places for them

Agree completely.

I’ve just posted a separate forum post having a bitch about the state of the documentiation

Happy to collaborate (as much as I can) but really need a decision on where/how documentation is going to be stored maintained before we do anything else.

[popey]

Sure. I appreciate you taking the time to get stuck in to fix the docs.

The fact remains docs.snapcraft.io is what’s linked from snapcraft.io, and is where we point developers. Until the “experimental” site Gustavo set becomes official (or not), docs go there imo.

Having them in multiple places is (as you point out in your other thread) a confusing mess for new people. I spent some time with new developers earlier this week and this problem came up as a significant blocker. Wrong docs, docs lacking detail and too many locations for docs were all cited as on-ramp issues we need to fix.

[bsutton]

@popey

The docs you’ve copied over are also needlessly over-complicated

I found the less complicated build process and have updated the doco.

[bsutton]

@popey

The fact remains docs.snapcraft.io is what’s linked

So how do I contribute to the original site?

[bashfulrobot]

Quick question here… in the past, I have run into errors when installing a snap inside LXD. Has something changed there? I seem to remember needing to install squashfuse (but still hitting mount errors).

I have mostly been building using the deb package inside the container. (Custom lxd profile that auto installs it).

[bsutton]

The discussion here is around building a snap using lxd as apposed to installing an existing snap into an lxd container.

I’ve not tried installing a snap into lxd so can’t help much on that front.

[niemeyer]

@popey

The fact remains docs.snapcraft.io is what’s linked from snapcraft.io, and is where we point developers. Until the “experimental” site Gustavo set becomes official (or not), docs go there imo.

@popey The chance of it not becoming official is low at this point. We have some pretty good appreciation for the idea so far, including other projects interested in using the same approach, and I don’t really expect we’ll ever achieve full consensus on something like this given the context (too many people involved). The main reason why we still have an experimental label at this point is precisely that we haven’t migrated all of the content, and this is something I’d really appreciate your help on.

Also, note that the problem you describe, of copying leaving stale docs, is exactly the case on some of the content in docs.snapcrat.io today. Much of the documentation we have in the new experimental site existed even before the docs site was in place, and the content was taken out of the wiki, modified, split out (argh), and then not updated of course.

Another interesting data point to observe here is: someone was willing to go over and improve the documentation of the project, unprompted, out of their own interest. And they have their documentation live, today, right now. If you see any way in which that documentation could be improved, you have access to it. Just edit it, and it will be live, today as well.

That’s how good collaboration feels like. Paraphrasing, it’s the opposite of “Don’t write instructions here. Don’t copy and improve old documentation. That’s over complicated.”.

@popey

I’m mid way through re-writing some of the pages and it would be great if we could work together on those docs rather than have 2 (now 3) places for them.

You could be doing that already, and we’ll appreciate your help. Otherwise, we’ll have to copy over the content until it’s a superset, and then we’ll replace the documentation part of the current site.

[hanishkvc]

Add lxd as a supplemental group rather than primary group

In order to allow using of lxd directly by the user instead of requiring to route/gate it thro sudo each time (including indirectly when running snapcraft cleanbuild which inturn uses lxd). It is currently suggested to do

$ sudo usermod -g lxd ${USER}

But this will change the primary group of the user, is it really what is required, because this will change the group owner ship of all files/directories in the users home directory to this new group lxd, which I don’t think is correct or warranted.

Shouldn’t this be

$ sudo usermod -a -G lxd ${USER}

such that the user gets added to lxd group, but his primary group assignment doesnt change.

NOTE: I had a scare of my life, when I in the middle of experimenting with snap packages and packaging, followed this document initially blindly (my bad, but one will expect the documentation to suggest sane settings by default, for good or bad), and then next day when I was doing something listed my home files and noticed that everything was group owned by lxd. Retracing the things which I did the previous day is what made me notice this potential wrong suggestion in the document.

A different (less intrusive) approach to persistant lxd based build setup

As documented currently,

using snapcraft cleanbuild keeps recreating the build time container again and again

However the alternative suggested i.e export SNAPCRAFT_BUILD_ENVIRONMENT=lxd snapcraft requires one to allow permission to map the users uid and gid into containers created/managed/used by lxd always (i.e editing of /etc/subgid and /etc/subuid) and not just for this particular build time container.

A alternate less intrusive (but requiring some additional manual steps currently) approach could be to

a) setup a build time container
lxc launch ubuntu:xenial snap-build-myapp
lxc start snap-build-myapp
lxc shell snap-build-myapp
snap install snapcraft --classic
su ubuntu
cd
mkdir snaps
mkdir snaps/myapp

b) copy the snap directory content from host to container using lxc file push

lxc file push snaps/myapp/snap/snapcraft.yaml snap-build-myapp/home/ubuntu/snaps/myapp/snap/
repeat for any additional support files

Or tar, push from host and untar in container.

c) run snapcraft in the container

d) pull the created snap into host from container

if the snapcraft team can use the above approach, then there is no need to do those intrusive /etc/subuid|subgid changes.

[Lin-Buo-Ren]

SNAPCRAFT_BUILD_ENVIRONMENT=lxd no longer supported in recent Snapcraft releases.

[jarl-dk]

I couldn’t agree more with the usermod change. I also just followed blindly. I discovered several months later and believed that my computer has been compromised. It took some time to investigate. Then I found this documentation and remembered that I tried this out months earlier.

If it really is necessary to change the primary group, then the descriptive text should be updated to reflect the command:

then you need to add your user account to the lxd group

should be updated to

then you need to change the primary login group of your user account to the lxd group

However, again, I don’t believe that is necessary

[Lin-Buo-Ren]

@hanishkvc @jarl-dk Thanks for escalating the issue, I’ve edited the post, please comment if it’s fine now.

[jarl-dk]

It must be capital G as $ sudo usermod -a -G lxd ${USER}
not $ sudo usermod -a -g lxd ${USER}

Jarl

[degville]

@jarl-dk

$ sudo usermod -a -G lxd ${USER}

Thanks! This is now fixed too.

[ijohnson]

This should be updated to reflect the new --use-lxd option to snapcraft, shouldn’t it? I.e. the section on building with bases should have explain how to use --use-lxd in addition to the current explanation of manually creating an LXD container.

[degville]

@ijohnson

This should be updated to reflect the new --use-lxd option to snapcraft, shouldn’t it?

You’re absolutely right. I’ve updated the doc and also added the corresponding release notes for the snapcraft update (3.4) that added --use-lxd. Thank you!

[rbasak]

@Lin-Buo-Ren

SNAPCRAFT_BUILD_ENVIRONMENT=lxd no longer supported in recent Snapcraft releases.

Seems to work with 4.6.2? Additionally I see for example that https://github.com/snapcore/action-build/blob/master/src/build.ts is using it. Can the current situation be documented please? It’s tedious to have to use --use-lxd all the time when changing the snapcraft invocation command line.