Building snaps without `snapd`

snapcraft
1

Right now it is impossible to build snaps with newer bases in Docker containers. First of all, the version of snapcraft shipped with latest Ubuntu is old.

$ docker run -it ubuntu:19.04 bash -c "apt-get -y update && apt-get -y install snapcraft && snapcraft --version"
...
snapcraft, version 2.43.1+18.10

Installing newer version of snapcraft from snap fails.

$ docker run -it ubuntu:19.04 bash -c "apt-get -y update && apt-get -y install snapd && snap install snapcraft && snapcraft --version"
...
error: cannot communicate with server: Post http://localhost/v2/snaps/snapcraft: dial unix /run/snapd.socket: connect: no such file or directory

As a result, no any docker based CI is able to build snaps with bases.

Is it possible to still build snaps without using this snapd daemon? In the end it is just a squashfs archive. Why snapcraft is following the docker anti-pattern for building images?

2

This isn’t entirety accurate. It is true that the version of snapcraft available as a Debian package in the Ubuntu archives is not sufficient to build snaps, but that’s expected since AFAIK the snapcraft team is only releasing updates to the snap version of snapcraft. In response to this, what I’ve been doing for my projects that need to build using docker containers is to follow the directions of the upstream snapcraft docker image and “install” the snapcraft and core snaps into an Ubuntu docker container of the appropriate base (i.e. for a core18 base snap use a docker base image of ubuntu:18.04) and then run snapcraft that way. The significant bit here being how you install the snap in the container. Luckily here, since snapcraft is a classic snap we don’t have to jump through many hoops - we just download the snap and extract it with unsquashfs and then just basically call the snapcraft binary in the extracted snap folder with some env variables set. The upstream dockerfile for snapcraft is the best example of this: https://github.com/snapcore/snapcraft/blob/master/docker/stable.Dockerfile

2 Likes
3

I created Docker image with Ubuntu 18.10 and latest snapcraft as you wrote, and it doesn’t work.

✗ podman run -v $PWD:/src:Z -w /src yakshaveinc/snapcraft:core18-edge snapcraft          
Locale not set! Snapcraft will temporarily use C.UTF-8
Using 'snapcraft.yaml': Project assets will be searched for from the 'snap' directory.
You need 'multipass' set-up to build snaps: https://github.com/CanonicalLtd/multipass/releases.

--destructive-mode fails too.

✗ podman run -v $PWD:/src:Z -w /src yakshaveinc/snapcraft:core18-edge snapcraft --destructive-mode -d   
Locale not set! Snapcraft will temporarily use C.UTF-8
Using 'snapcraft.yaml': Project assets will be searched for from the 'snap' directory.
Failed to get information for snap 'core18': could not connect to 'http+unix://%2Frun%2Fsnapd.socket/v2/snaps/core18'.

The image is build with this script:

git clone --depth=100 https://github.com/snapcore/snapcraft .
sed -i 's/xenial/bionic/g' docker/edge.Dockerfile && echo docker/edge.Dockerfile
docker build -f docker/edge.Dockerfile docker
4

If you are going to use docker, you will need to also do the same dance to fetch core18 and unpack it as done in the Dockerfile for core.

5

all these errors stem from snapd not yet running while your script moves on and tries to run snap commands …

from my experience with

it can take up to 60 seconds after you installed snapd to have it actually run inside the docker container, try to give it that time (add some sleep or whatever) and see if it solves the issue. (note, the above is hackish and old (against 16.04), but i suspect the issues i did hit back then still apply)

1 Like
6

Ok. I think I cheated myself. The commands above are from podman - a daemonless replacement for Docker. I fix commands to avoid future confusion

With plain docker I could build my snap on my local machine, but that creates root owned dirs in my folder.

✗ find . -group root
./parts
./parts/yakshaveinc
./parts/yakshaveinc/state
./parts/yakshaveinc/state/stage
./parts/yakshaveinc/state/build
./parts/yakshaveinc/state/prime
./parts/yakshaveinc/state/pull
./parts/yakshaveinc/install
./parts/yakshaveinc/src
./parts/yakshaveinc/build
./parts/.snapcraft_global_state
./stage
./prime
./prime/meta
./prime/meta/snap.yaml

which means that it probably starts snapd process on my machine under root. Which means it fails to run inside container, which means it will fail on any container-based CI.

So, the question is the same - how to build snaps without snapd? Why it is not possible?

7

Why do you need snapd? You can build without snapd as I explained by just doing the same as the upstream Dockerfile for snapcraft and download snapcraft, core, core18 snaps into the container and run those directly. This doesn’t need root, nor does it need snapd.

8

I could successfully build snap from Travis CI https://travis-ci.org/yakshaveinc/linux/builds/528809947#L585

I still don’t really understand what is going on there. Starting from original instructions.

curl -L $(curl -H 'X-Ubuntu-Series: 16' 'https://api.snapcraft.io/api/v1/snaps/details/core' | jq '.download_url' -r) --output core.snap
  1. ^ Downloads core.snap from snapcraft API
mkdir -p /snap/core && unsquashfs -d /snap/core/current core.snap && rm core.snap
  1. ^ Unpack core.snap to /snap/core/current 
curl -L $(curl -H 'X-Ubuntu-Series: 16' 'https://api.snapcraft.io/api/v1/snaps/details/snapcraft?channel=stable' | jq '.download_url' -r) --output snapcraft.snap && \
  mkdir -p /snap/snapcraft && unsquashfs -d /snap/snapcraft/current snapcraft.snap
  1. ^ Downloads snapcraft from API and “install” it the same way
COPY bin/snapcraft-wrapper /snap/bin/snapcraft
  1. ^ Add some hacking to make unpacked snapcraft run is familiar environment

What I don’t understand is that I don’t download core18 and yet snap builds successfully. If it detected that it runs on Ubuntu 18.04, then it would be nice to see this as a message.

9

I also don’t understand why podman fails if the image doesn’t require root privileges.

10

Well it probably doesn’t fail because the rootfs from the core18 snap that your snap runs in is a subset of the rootfs you have in your ubuntu 18.04 container, so the snap build will say it works, but the potential issue is that there are packages in your container that snapcraft will assume are also present in the core18 snap but actually aren’t. I’m not sure how best to guard against this except actually testing the resultant snap to see what happens.

Can you post an example of what output is produced when this process is used with podman and snapcraft fails?

11

All failures in my second post are from podman Building snaps without `snapd` I use podman for everything, because it runs containers without daemons, root access and creating those root owned files in my dir.

I’ve just realized how many hacks are used to get the core18 build using docker container.

  1. snapcraft in apt repository is outdated - 2.x - it is impossible to use that for core18 builds
  2. snapcraft from snap uses core from Ubuntu 16.04 and even if executed from Ubuntu 18.04, it can not access its files necessary for core18
  3. When I built the image yakshaveinc/snapcraft:core18-edge with Ubuntu 18.04 and native snapcraft, then finally snapcraft could access files necessary for core18 from the main system, but… it was not enough

Running container with native snapcraft on Ubuntu 18.04 worked only in Docker:

✗ docker run -v $PWD:/src:Z -w /src yakshaveinc/snapcraft:core18-edge snapcraft                   
Locale not set! Snapcraft will temporarily use C.UTF-8
Using 'snapcraft.yaml': Project assets will be searched for from the 'snap' directory.
The following snaps are required but not installed as snapcraft is running inside docker: core18.
Please ensure the environment is properly setup before continuing.
Ignore this message if the appropriate measures have already been taken
Pulling yakshaveinc 
Building yakshaveinc 
Staging yakshaveinc 
Priming yakshaveinc 
'confinement' property not specified: defaulting to 'strict'
'grade' property not specified: defaulting to 'stable'
Snapping 'yakshaveinc' ...

Snapped yakshaveinc_eternal_amd64.snap

But fails with another engine for running Docker containers:

✗ podman run -v $PWD:/src:Z -w /src yakshaveinc/snapcraft:core18-edge snapcraft
Locale not set! Snapcraft will temporarily use C.UTF-8
Using 'snapcraft.yaml': Project assets will be searched for from the 'snap' directory.
You need 'multipass' set-up to build snaps: https://github.com/CanonicalLtd/multipass/releases.

even with explicit --destructive-mode, but with different error

✗ podman run -v $PWD:/src:Z -w /src yakshaveinc/snapcraft:core18-edge snapcraft --destructive-mode
Locale not set! Snapcraft will temporarily use C.UTF-8
Using 'snapcraft.yaml': Project assets will be searched for from the 'snap' directory.
Failed to get information for snap 'core18': could not connect to 'http+unix://%2Frun%2Fsnapd.socket/v2/snaps/core18'.

And the reason for that is hack no.4

  1. Explicit check for running in Docker container - see docker run output above

The following snaps are required but not installed as snapcraft is running inside docker: core18. Please ensure the environment is properly setup before continuing. Ignore this message if the appropriate measures have already been taken

I tried to fix my core18 builds in TravisCI for over 3 months and this week we’ve got a long holiday here, in Belarus and I could concentrate on cleaning my own technical debts. I am so happy that builds have started to work finally - https://travis-ci.org/yakshaveinc/linux/builds/528809947#L585 and I am even more happy to understand why. :slight_smile: Thanks for guiding me, Mr.@ijohnson.

What can help to solve issue with podman. Replacing Docker-specific check to some generic container check. More explicit debug output how snapcraft chooses environment where the snap is built and why. Explicit flag to run without wrappers. I thought that --destructive-mode is that flag, but what I still don’t understand - why running with snapcraft --destructive-mode with podman (for core18, snapcraft installed natively n Ubuntu 18.04) fails with the error of accessing snapd?

1 Like
12

It looks like all snapcraft does to identify if it is running in a docker container is check for a .dockerenv file: https://github.com/snapcore/snapcraft/blob/5bca5edee51625d80a57b0495d550100c7795f59/snapcraft/internal/common.py#L127-L128

Perhaps you could either manually create this file in podman (maybe podman should create this file if it’s supposed to look like docker?), or you could submit a patch to snapcraft such that snapcraft would be able to detect it’s running inside a podman container and operate the same as if it’s running in a docker container?

I assume because if snapcraft doesn’t think it’s running inside a container it assumes it’s running on some version of linux (either in a VM or otherwise natively) where it can talk to snapd to install base snaps or other snaps from i.e. build-snaps, etc. from the snapcraft.yaml (for example the go plugin will now by default install go using the go snap). I thought that there was some environment variable you could provide to snapcraft to inform it what kind of environment it was running in like SNAPCRAFT_BUILD_ENVIRONMENT or something - I seem to remember @Lin-Buo-Ren mentioning this somewhere somewhat recently) that either can tell snapcraft it’s running inside a container and as such has no snapd to talk to, or could be extended such that you can use that environment variable (or maybe a new one similar to that) to tell snapd it’s running inside a container and has no snapd to talk to. I imagine from the existence of the is_docker_instance function from above that the env var doesn’t currently work like that, but I could see a case that it should work like that.

13

Whoa. Am a bit shocked at how old the 18.04 snapcraft is! We should SRU a fresh version for sure.

1 Like
14

I placed .dockerenv files inside the container image and was able to run snapcraft with podman from Fedora.

podman run -it -v $PWD:/src:Z -w /src yakshaveinc/snapcraft:core18 snapcraf

The image is publicly available from https://cloud.docker.com/u/yakshaveinc/repository/docker/yakshaveinc/snapcraft

Maybe checking for explicit flag inside the container is better than relying on Docker to create this file. It doesn’t look like an industry standard across different Linux container wrappers.

I asked for a proper way to detect podman here https://github.com/containers/libpod/issues/3586

I see no way why not let snapcraft operate without snapd daemon for building purposes. Downloading, unsquashing and squashing files doesn’t need privileged deamons.

15

From your post there, it seems that /run/.containerenv is a guaranteed, documented way to tell that it’s running inside a container, perhaps you could submit a PR to snapcraft to detect that file as well as the .dockerenv file.

The reason for this is because when using build-snaps, the snaps are meant to be installed and usable by the commands specified in the snapcraft.yaml and for strictly confined snaps, there is a significant amount of setup to allow strictly confined snaps to run, it’s not just downloading and unsquashing files.

16

Done.

17

Could you give an example?

18

The easiest example would be the root filesystem / mount namespace setup that is performed by various mounts, which require real root on the outside and also CAP_SYS_ADMIN, which is denied to most containers by default. Trying to replicate all of the mount namespace setup is a very non-trivial amount of work, and if you tried to just run the executables inside snaps without any of that, many of them would not work at all since they would be expecting to have their core or core18 base setup properly.
Additionally, you have things like snapd config features which many snaps use in the form of snapctl get KEY or snapctl set KEY=VALUE, etc. which won’t work without having snapd running.