Build on LXD


#1

Building snaps within an isolated environment, such as LXD, has several advantages over a native build:

  • containers keep your host system clear of your new snap’s dependencies
  • your host system can’t expose conflicting libraries to your new snap build

Read on to discover how to build snaps with LXD.

Install LXD

Recent non-desktop versions of Ubuntu install LXD by default - you can check whether it is installed with the following command:

$ lxd version
3.7

The easiest way to add LXD to your system is via its snap:

$ sudo snap install lxd

Now initialise LXD with the following command, accepting all the default options unless you have specific requirements:

$ sudo lxd init

ⓘ If the system you are installing LXD onto is using a network with a 10.x.x.x subnet then network create may fail. Step through the following to resolve this.

Manually create a network on a 10.x.x.x subnet

If you try to run lxd init on a system that is connected to a network with a 10.x.x.x subnet, then the final step of the Iinit* may fail with the following error:

Error: Failed to create network 'lxdbr0': Failed to automatically find an unused IPv4 subnet, manual configuration required

To resolve this issue, you must manually create the network with the following command:

$ sudo lxc network create lxdbr0 ipv4.address=10.0.3.1/24 ipv4.nat=true

You can then re-run lxd init. When you are prompted to create a new network bridge you must respond no.

Would you like to create a new network bridge (yes/no) [default=yes]? no

Group permissions

If you want to build snaps as a non-root user, which is advised, then you need to add your user account to the lxd group:

$ sudo usermod -g lxd ${USER}

You now need to either restart your session, reboot your computer, or use newgrp to acquire the new group assignment:

$ newgrp lxd

ⓘ The newgrp command will start a new sub-shell (shell within a shell) with the new lxd group assigned.

Building

ⓘ If snapcraft is not yet installed, enter sudo snap install snapcraft --classic.

To build your project using lxd run:

$ cd $HOME/my-snap-project
$ snapcraft cleanbuild

If the build fails, add the --debug option to the snapcraft cleanbuild command. This will drop you into a shell running within the LXD environment that just attempted to build your snap, allowing you to examine where the issue may be.

Experimental: persistent containers

When building with LXD, the old container is discarded. This means that LXD needs to build a new environment, each time you build your snap. This is acceptable with small snaps, or for a final build, but can be problematic during development.

For example, during development, you might normally use:

$ snapcraft
$ snap try prime

The ‘try’ method allows you to directly edit files in a snap’s prime directory without having to re-install or re build after each change. However, when using cleanbuild, this feature is not supported.

But there is an experimental feature that uses LXD in a different way, allowing you to retain intermediate files, and the container:

$ export SNAPCRAFT_BUILD_ENVIRONMENT=lxd 
$ snapcraft

If you get the following error:

Failed to start container: the local folder could not be mounted into the container. 

Add a line containing root:1000:1 to the following files:

  • /etc/subuid
  • /etc/subgid

You may need to log out and back in again before reattempting the build. If successful, you can now try the snap, make changes, and rebuild quickly:

$ snap try prime

The environment variable SNAPCRAFT_BUILD_ENVIRONMENT instructs snapcraft to use experimental LXD integration to build the snap in a persistent LXD container. This container remains after the build has finished, and retains intermediate build files to speed up rebuilds.


Error downloading stage packages for part 'desktop-qt5': The package 'appmenu-qt5' was not found
Snapcraft overview
HTML5 game without electron
Snapcraft installing golang 1.6 when 1.10.1 is installed
How to Create an LXD Container for Snap Development
Snap Documentation
Permission denied when building with snapcore/snapcraft
Do you really need LXD?
Snapcraft failed to build due to incorrect glibc version
Build on Docker
Improving the documentation
#2

It looks like you’ve copy/pasted the docs directly from docs.snapcraft.io to here. I’d advise against that. We’re currently in the process of re-writing some of the docs there and copy/pasting old copies around the place will leave stale docs here.

The docs you’ve copied over are also needlessly over-complicated (which is why I’m mid way through re-writing them) and will confuse new users. So I’d recommend we simplify them in one place for now.


#3

It wasn’t a straight copy paste as the old version was out of date.

I’ve done a re-write (based on my understanding) and I’m in the process of testing the steps.

If there are simpler steps then let me know.

The docker instructions were a copy/paste.

Were are you re-writing the docs?
I thought the process was via the forums?

And just some feedback.

One of the key problems with the current docs is a lack of details.
I agree we need a broad overview but we really need to allow users to drill down into more detail.
There is far too much that is left unsaid in the name of keeping things simple.
I haven’t found the current docs very useful specifically because of this problem.


#4

The docs are hosted at docs.snapcraft.io - the source for them is on github. We welcome changes to the docs. I’m mid way through re-writing some of the pages and it would be great if we could work together on those docs rather than have 2 (now 3) places for them. While Gustavo has a documentation experiment website up, the official docs have been and still are at docs.snapcraft.io.


#5

Agree completely.

I’ve just posted a separate forum post having a bitch about the state of the documentiation :slight_smile:

Happy to collaborate (as much as I can) but really need a decision on where/how documentation is going to be stored maintained before we do anything else.


#6

Sure. I appreciate you taking the time to get stuck in to fix the docs.

The fact remains docs.snapcraft.io is what’s linked from snapcraft.io, and is where we point developers. Until the “experimental” site Gustavo set becomes official (or not), docs go there imo.

Having them in multiple places is (as you point out in your other thread) a confusing mess for new people. I spent some time with new developers earlier this week and this problem came up as a significant blocker. Wrong docs, docs lacking detail and too many locations for docs were all cited as on-ramp issues we need to fix.


#7

I found the less complicated build process and have updated the doco.


#8

So how do I contribute to the original site?


#9

Quick question here… in the past, I have run into errors when installing a snap inside LXD. Has something changed there? I seem to remember needing to install squashfuse (but still hitting mount errors).

I have mostly been building using the deb package inside the container. (Custom lxd profile that auto installs it).


#10

The discussion here is around building a snap using lxd as apposed to installing an existing snap into an lxd container.

I’ve not tried installing a snap into lxd so can’t help much on that front.


#13

@popey The chance of it not becoming official is low at this point. We have some pretty good appreciation for the idea so far, including other projects interested in using the same approach, and I don’t really expect we’ll ever achieve full consensus on something like this given the context (too many people involved). The main reason why we still have an experimental label at this point is precisely that we haven’t migrated all of the content, and this is something I’d really appreciate your help on.

Also, note that the problem you describe, of copying leaving stale docs, is exactly the case on some of the content in docs.snapcrat.io today. Much of the documentation we have in the new experimental site existed even before the docs site was in place, and the content was taken out of the wiki, modified, split out (argh), and then not updated of course.

Another interesting data point to observe here is: someone was willing to go over and improve the documentation of the project, unprompted, out of their own interest. And they have their documentation live, today, right now. If you see any way in which that documentation could be improved, you have access to it. Just edit it, and it will be live, today as well.

That’s how good collaboration feels like. Paraphrasing, it’s the opposite of “Don’t write instructions here. Don’t copy and improve old documentation. That’s over complicated.”.

You could be doing that already, and we’ll appreciate your help. Otherwise, we’ll have to copy over the content until it’s a superset, and then we’ll replace the documentation part of the current site.


#14

Add lxd as a supplemental group rather than primary group

In order to allow using of lxd directly by the user instead of requiring to route/gate it thro sudo each time (including indirectly when running snapcraft cleanbuild which inturn uses lxd). It is currently suggested to do

$ sudo usermod -g lxd ${USER}

But this will change the primary group of the user, is it really what is required, because this will change the group owner ship of all files/directories in the users home directory to this new group lxd, which I don’t think is correct or warranted.

Shouldn’t this be

$ sudo usermod -a -G lxd ${USER}

such that the user gets added to lxd group, but his primary group assignment doesnt change.

NOTE: I had a scare of my life, when I in the middle of experimenting with snap packages and packaging, followed this document initially blindly (my bad, but one will expect the documentation to suggest sane settings by default, for good or bad), and then next day when I was doing something listed my home files and noticed that everything was group owned by lxd. Retracing the things which I did the previous day is what made me notice this potential wrong suggestion in the document.

A different (less intrusive) approach to persistant lxd based build setup

As documented currently,

using snapcraft cleanbuild keeps recreating the build time container again and again

However the alternative suggested i.e export SNAPCRAFT_BUILD_ENVIRONMENT=lxd snapcraft requires one to allow permission to map the users uid and gid into containers created/managed/used by lxd always (i.e editing of /etc/subgid and /etc/subuid) and not just for this particular build time container.

A alternate less intrusive (but requiring some additional manual steps currently) approach could be to

a) setup a build time container
lxc launch ubuntu:xenial snap-build-myapp
lxc start snap-build-myapp
lxc shell snap-build-myapp
snap install snapcraft --classic
su ubuntu
cd
mkdir snaps
mkdir snaps/myapp

b) copy the snap directory content from host to container using lxc file push

lxc file push snaps/myapp/snap/snapcraft.yaml snap-build-myapp/home/ubuntu/snaps/myapp/snap/
repeat for any additional support files

Or tar, push from host and untar in container.

c) run snapcraft in the container

d) pull the created snap into host from container

if the snapcraft team can use the above approach, then there is no need to do those intrusive /etc/subuid|subgid changes.


#15

SNAPCRAFT_BUILD_ENVIRONMENT=lxd no longer supported in recent Snapcraft releases.