Per project containers

snapcraft
1

Right now there’s two ways to use snapcraft:

  • Run snapcraft to build natively on the host machine - some features only work if the host is Ubuntu (based).
  • Use cleanbuild which creates a temporary LXD container - the only supported workflow is equivalent to running “snapcraft” in a clean environment where everything except for the resulting .snap is discarded afterwards.

I’m introducing a third option:

  • Allow executing all snapcraft commands in a persistent LXD container that is automatically created and mounts the project folder instead of using a one-off tarball.
  • The host isn’t altered in any way and it doesn’t matter if it’s Ubuntu or something else.
  • Iterative development becomes feasible even if the container is on another machine in the cloud.
  • The use of containers is opaque, the workflow is the same as native.

For starters the new mode is hidden behind an environment variable - eventually there will be a prompt to ask the user if a container should be used for everything.

SNAPCRAFT_CONTAINER_BUILDS=1 snapcraft

6 Likes
2

Here’s one of the PRs that will get this going generally: https://github.com/snapcore/snapcraft/pull/1263

3

It is worthy to note that we only have this support sort of ready for when the lxd remote is local. We need to discuss with @tvoss or @stgraber on how to deal with real remotes. The shuffled options we threw to the air were:

  • ssh fuse
  • nfs

I personally prefer the former given the ubiquity even if it means a performance hit.

1 Like
4

I was talking to @stgraber and he’s suggested sshfs. I started looking into it and my primary concern to start with was having to setup something on the host that couldn’t be easily automated or undone; there seems to be a way, though, to ssh into the container with sftp-server on the machine running snapcraft. On the other hand nfs seems to require setup on the host and I’ve read about kernel limitations requiring a priviledged container.

6

I realize I never posted my findings from looking at existing solutions here.

7

When using this option, I could see that the lxd container was stopped. But on the next run (without closing the terminal), it spun up a new container. How do you leverage the previous container?

My steps were:

SNAPCRAFT_CONTAINER_BUILDS=1
snapcraft cleanbuild

Then on rerun, simply:

snapcraft cleanbuild

Thanks!

8

Don’t use cleanbuild, just snapcraft. By saying cleanbuild you’re actually instructing snapcraft to create a clean container every time.

9

Yeah, I was trying to keep my host system clean (installing deps, etc).

10

What SNAPCRAFT_CONTAINER_BUILDS=1 does is it enables regular commands to create a LXD container that’s persistent. So you can do snapcraft pull or snapcraft clean -s build mypart and it works inside the container (note: snapcraft update doesn’t work yet).

11

Thanks for clarification!

12

For those following this topic, there’s more improvements on the way:

13

Just came across this. This is SO USEFUL for guys not building on a LTS, e.g. on 17.10 and facing library issues. Why isn’t this already part of the documentation?

1 Like
14

There are some instructions, however not towards this specific way.

I use the lxd method.

1 Like
15

We don’t want to document this specifically since it’s a feature flag precisely to not make it official while we test. That said, it’s been a year, so why is this still behind an ugly environment feature-flag instead of being a polished experience, and perhaps even the default way most people are doing it nowadays?

Before we go there though, I’ve been out of touch with the progress on cross-distribution and multi-base progress, which is actually closely related to this topic too. Can we please meet to talk both about all of these topics?

A hangout would be best, including @sergiusens, @evan, and @kalikiana at least.

3 Likes
16

Would be nice if it could become the default way for snapcraft since it is less error-prone than building on non LTS releases/unclean installations which again results in a lot of questions related to local OS problems.

Specifically, I use both a 16.04 and 17.10 currently and some apps do not build at 17.10 due to some library conflicts. Simply building everything in the LTS as it is also done when deploying via travis (docker is used there) would help a lot.

In this regard, one might question why docker is used on travis and LXD for local building. There should be an agreement on using one service for both.

1 Like
17

Why would you narrow it down to only one? There’s use cases for both. For example on a local build on a laptop for an individual who has not used Docker before… It may be more approachable. Or at least perceived more approachable. Ultimately they’re both just containers anyways. And in my opinion if even one of the Technologies is seen as easier to get started with… It makes snaps more approachable. Although ultimately if the documentation is good I do admit that it wouldn’t be a big deal either way. Just a thought.

18

Yeah I forget you have to snap install lxd and lxd init for SNAPCRAFT_CONTAINER_BUILDS=1 snapcraft to work. If this can’t be automated then this seems unwise to make default, though it’s so good that really it should be default! Maybe changing it from a feature flag to snapcraft lxdbuild or snapcraft persistbuild or something (like snapcraft cleanbuild) would be good and for snapcraft to tell the user to run snap install lxd and lxd init as appropriate. Also a flag for using a different container (and, thus, different libraries) to Ubuntu 16.04, e.g. --ubuntu-17.10 or --debian-9 or something.

19

While all that is good, I was under the impression that by best practice, a container should only be built against 16.04. (I thought).

20

So I just tried this option but got an error:

Note: I can snapcraft cleanbuild on this system without an issue.

x11-common set to automatically installed. Traceback (most recent call last): File “/snap/snapcraft/1094/bin/snapcraft”, line 11, in load_entry_point(‘snapcraft==2.39’, ‘console_scripts’, ‘snapcraft’)() File “/snap/snapcraft/1094/usr/lib/python3.6/site-packages/pkg_resources/init.py”, line 565, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File “/snap/snapcraft/1094/usr/lib/python3.6/site-packages/pkg_resources/init.py”, line 2631, in load_entry_point return ep.load() File “/snap/snapcraft/1094/usr/lib/python3.6/site-packages/pkg_resources/init.py”, line 2291, in load return self.resolve() File “/snap/snapcraft/1094/usr/lib/python3.6/site-packages/pkg_resources/init.py”, line 2297, in resolve module = import(self.module_name, fromlist=[‘name’], level=0) File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/cli/main.py”, line 43, in run(prog_name=‘snapcraft’) File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 722, in call return self.main(*args, **kwargs) File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 697, in main rv = self.invoke(ctx) File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 1066, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 535, in invoke return callback(*args, **kwargs) File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/cli/lifecycle.py”, line 139, in snap project_options, directory=directory, output=output) File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_packer.py”, line 46, in snap execute(‘prime’, project_options) File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py”, line 70, in execute os.makedirs(constants.SNAPCRAFT_INTERNAL_DIR, exist_ok=True) File “/snap/snapcraft/1094/usr/lib/python3.6/os.py”, line 220, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: ‘snap/.snapcraft’ Stopping local:snapcraft-tomcat-with-ssl

The permissions for the snap directory are:

rwxr-xr-x 4 bsutton lxd      4096 Feb 23 01:50 ./
drwxr-xr-x 3 bsutton lxd      4096 Feb 23 01:24 ../
drwxr-xr-x 4 bsutton lxd      4096 Feb 23 01:24 snap/
-rw-r--r-- 1 bsutton lxd      2560 Feb 23 01:24 snapcraft.yaml

The snap directory contains:

ls -l snap
drwxr-xr-x 2 bsutton lxd 4096 Feb 23 01:24 hooks
drwxr-xr-x 3 bsutton lxd 4096 Feb 23 01:24 src

So I’m guessing the container was trying to write as ‘lxd’ to the snap folder which is why it failed. So I changed the snap folder to: 775 So that lxd could write to the folder, but that also failed. I had to change the permission to 777 before it was able to write to the snap directory which just seems wrong.

21

So having solved the snap 777 problem (or rather worked around it).

I did another build but this time I’m getting a new file permission error:

Mounting /home/bsutton/git/tomcat-with-ssl-snap into container
Device /root/build_tomcat-with-ssl added to snapcraft-tomcat-with-ssl
Traceback (most recent call last):
File “/snap/snapcraft/1094/bin/snapcraft”, line 11, in
load_entry_point(‘snapcraft==2.39’, ‘console_scripts’, ‘snapcraft’)()
File “/snap/snapcraft/1094/usr/lib/python3.6/site-packages/pkg_resources/init.py”, line 565, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File “/snap/snapcraft/1094/usr/lib/python3.6/site-packages/pkg_resources/init.py”, line 2631, in load_entry_point
return ep.load()
File “/snap/snapcraft/1094/usr/lib/python3.6/site-packages/pkg_resources/init.py”, line 2291, in load
return self.resolve()
File “/snap/snapcraft/1094/usr/lib/python3.6/site-packages/pkg_resources/init.py”, line 2297, in resolve
module = import(self.module_name, fromlist=[‘name’], level=0)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/cli/main.py”, line 43, in
run(prog_name=‘snapcraft’)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 722, in call
return self.main(*args, **kwargs)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 697, in main
rv = self.invoke(ctx)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/click/core.py”, line 535, in invoke
return callback(*args, **kwargs)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/cli/lifecycle.py”, line 139, in snap
project_options, directory=directory, output=output)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_packer.py”, line 46, in snap
execute(‘prime’, project_options)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py”, line 79, in execute
_Executor(config, project_options).run(step, part_names)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py”, line 185, in run
self._run_step(step, part, part_names)
File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py”, line 215, in run_step
getattr(part, 'prepare{}’.format(step))()
File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/internal/pluginhandler/init.py”, line 263, in prepare_pull
self.makedirs()
File “/snap/snapcraft/1094/lib/python3.6/site-packages/snapcraft/internal/pluginhandler/init.py”, line 144, in makedirs
os.makedirs(d, exist_ok=True)
File “/snap/snapcraft/1094/usr/lib/python3.6/os.py”, line 210, in makedirs
makedirs(head, mode, exist_ok)
File “/snap/snapcraft/1094/usr/lib/python3.6/os.py”, line 210, in makedirs
makedirs(head, mode, exist_ok)
File “/snap/snapcraft/1094/usr/lib/python3.6/os.py”, line 220, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: ‘/root/build_tomcat-with-ssl/parts’

I tried setting the parts directory permissions to 777 (on the host) to get around this problem. However the error persisted:

ll /root
total 36
drwxrwxrwx 6 root root 4096 Feb 23 03:53 ./
drwxr-xr-x 24 root root 4096 Feb 23 01:21 …/
drwxrwxrwx 3 bsutton lxd 4096 Feb 23 03:53 build_tomcat-with-ssl/
drwxr-xr-x 4 root root 4096 Feb 23 03:12 snap/

ll /root/build_tomcat-with-ssl/
drwxrwxrwx 2 bsutton lxd 4096 Feb 23 03:53 parts/

ll /root/build_tomcat-with-ssl/parts/
drwxrwxrwx 2 bsutton lxd 4096 Feb 23 03:53 ./
drwxrwxrwx 3 bsutton lxd 4096 Feb 23 03:53 …/