On https://snapcraft.io/docs/build-on-docker the example Dockerfile for building for core18 first references ubuntu:bionic, but then in the middle is based on ubuntu:xenial before the copy commands
The script postet by @kyrofa is more like I expected. This script additionally downloads the core18 snap, which the documentation does not do.
I propose the following changes to the documentation
FROM ubuntu:bionic as builder
# Grab dependencies
RUN apt-get update
RUN apt-get dist-upgrade --yes
RUN apt-get install --yes \
curl \
jq \
squashfs-tools
# Grab the core snap from the stable channel and unpack it in the proper place
RUN curl -L $(curl -H 'X-Ubuntu-Series: 16' 'https://api.snapcraft.io/api/v1/snaps/details/core' | jq '.download_url' -r) --output core.snap
RUN mkdir -p /snap/core
RUN unsquashfs -d /snap/core/current core.snap
+# Grab the core18 snap from the stable channel and unpack it in the proper place
+RUN curl -L $(curl -H 'X-Ubuntu-Series: 16' 'https://api.snapcraft.io/api/v1/snaps/details/core18' | jq '.download_url' -r) --output core18.snap
+RUN mkdir -p /snap/core18
+RUN unsquashfs -d /snap/core18/current core18.snap
+
# Grab the snapcraft snap from the stable channel and unpack it in the proper place
RUN curl -L $(curl -H 'X-Ubuntu-Series: 16' 'https://api.snapcraft.io/api/v1/snaps/details/snapcraft?channel=stable' | jq '.download_url' -r) --output snapcraft.snap
RUN mkdir -p /snap/snapcraft
RUN unsquashfs -d /snap/snapcraft/current snapcraft.snap
# Create a snapcraft runner (TODO: move version detection to the core of snapcraft)
RUN mkdir -p /snap/bin
RUN echo "#!/bin/sh" > /snap/bin/snapcraft
RUN snap_version="$(awk '/^version:/{print $2}' /snap/snapcraft/current/meta/snap.yaml)" && echo "export SNAP_VERSION=\"$snap_version\"" >> /snap/bin/snapcraft
RUN echo 'exec "$SNAP/usr/bin/python3" "$SNAP/bin/snapcraft" "$@"' >> /snap/bin/snapcraft
RUN chmod +x /snap/bin/snapcraft
# Multi-stage build, only need the snaps from the builder. Copy them one at a
# time so they can be cached.
-FROM ubuntu:xenial
+FROM ubuntu:bionic
COPY --from=builder /snap/core /snap/core
+COPY --from=builder /snap/core18 /snap/core18
COPY --from=builder /snap/snapcraft /snap/snapcraft
COPY --from=builder /snap/bin/snapcraft /snap/bin/snapcraft
# Generate locale
RUN apt-get update && apt-get dist-upgrade --yes && apt-get install --yes sudo locales && locale-gen en_US.UTF-8
# Set the proper environment
ENV LANG="en_US.UTF-8"
ENV LANGUAGE="en_US:en"
ENV LC_ALL="en_US.UTF-8"
ENV PATH="/snap/bin:$PATH"
ENV SNAP="/snap/snapcraft/current"
ENV SNAP_NAME="snapcraft"
ENV SNAP_ARCH="amd64"
Thank you for proposing those changes! I’ve merged them into the documented script.
I also added snapd as a dependency because, while testing, the image build failed with A tool snapcraft depends on could not be found: ‘snap’. I believe snapcraft now requires snapd externally.
After some internal discussion, I updated the sample Dockerfile to add Snap-CDN: none HTTP headers in order to work properly in the Launchpad build farm.
Thanks for this helpful thread & info.
I tried out the build in Docker and that absolutely works fine.
However, the point raised by @kyrofa is still valid: There is a prebuilt Docker image on Dockerhub from the organisation “snapcraft”, which contains images that are hopelessly outdated:
snapcraft’s Profile | Docker Hub
And there is a Dockerhub org “snapcore” which does seem to contain an up to date Docker image that matches the current Dockerfile from the snapcraft repo: snapcore’s Profile | Docker Hub
Is there a way we could improve this? Could we link from these docs directly to the correct Dockerhub? That would help save people wanting to use Docker some time.
FYI, an anecdote, but I’ve had to move away from docker entirely because snapcraft’s core plugins are making wider use of build-snaps, which don’t work on docker. Tread carefully.
Thanks for the heads-up. This got me thinking:
wouldn’t you agree that “limitations of using Docker for building snaps” would be an interesting paragraph to mention on a “Build on Docker” doc page too?
“terrible hacks”? I don’t agree with that sentiment. Yes it’s a hack, but suggesting that it is “terrible” will put people off using the images when they work perfectly fine.
The only scenario that they don’t work with currently is when you’re running them through the docker snap because the apparmor rules of the snap get in the way of the nested snapd instance.
Please allow me to get into this:
@lucyllewy is there somewhere I can read why you created your Docker that way and what problems you evade/fix? Again, with the mindset of expanding documentation, this is very useful.
From what I understand, the limitation of Docker is that it doesn’t contain snapd, which is needed for the build-snaps plugin. Snapd depends on systemd, which isn’t there by default, so you need a similar hack as in WSL to enable it?
Is that a correct?
Even if Docker can’t be an officially supported way of building snaps, and we have to add that big disclaimer on the docs page, offering a Docker image that actually works 99.9% of the time is still a major asset in my opinion. Not everyone can just spin up multipass VMs “just to try out a snapcraft build”. Docker and LXD are much more lightweight dependencies than multipass.
Just a small question: in your https://github.com/diddlesnaps/snapcraft-container/blob/master/entrypoint.sh , the entrypoint of your snapcraft Docker, you mount the securityfs, which isn’t enabled by default in the WSL kernel. Just double checking, is this needed? Cause my first attempt failed on that, and switching to your kernel config solved that issue.
In regards to the previous discussion, your snapcraft-container is OK, but, how would one put it… not intuitive.
For now I would propose adding the disclaimer somewhere in the documentation about Docker building, that with the current proposed steps there are limitations and that only multipass & LXE provide a 100% reliable result. Either on this docs page, or in a README file in the Docker folder of the snapcraft repo.
If that is ok for you guys, I will attempt to do just that.
The securityfs is required IF your kernel has apparmor enabled. Without apparmor it is not needed.
Rather than state “don’t use this” because “it’s not 100% reliable” please file issues that highlight the problems so that I can fix them. My goal is for this method to be as reliable as Multipass or LXD or Build Service builds. If it is currently not reliable, I need to know about it so that I can fix it.
Note that the only perspective I’m taking here is from a starting user trying out this method.
Please understand me correctly, I found your Docker image to build snaps reliable. But it is not intuitive.
Compare it to LXD. In LXD you can just spin up a container, install ubuntu & snapcraft and call snapcraft --destructive-mode . It is the equivalent of using --use-lxd in the snapcraft command. Very easy.
This is not the case in Docker. You have to fabricate a very custom Docker image to do this kind of stuff. The current docs mention a simple Dockerfile, but the resulting Docker container cannot build all snaps. So a disclaimer is needed right now, because there is information there that is just wrong.
If we can manage to get a build Docker to a point where a standard user can read it and understand what is going on, that would be ideal. The alternative is that the maintenance of that (more complex) Dockerfile is done in the snapcraft repo, and then it is an official way of building that is supported by the whole community. This is also something that is possible.
Thanks, can you add this exact line as a comment in the entrypoint.sh above the mount command?
I’m not sure how running the container image is more difficult. Can you elaborate some more about how it’s more difficult for you? It should be a simple case of executing this where you’d normally execute snapcraft:
docker run --rm -it --privileged -v $PWD:/data -w /data diddledan/snapcraft:core22
This is a standard docker invocation that should be understandable to anyone with experience of running containers in Docker.
Also, it should be noted that, the primary purpose of these container images is for CI/CD pipelines, where I’m maintaining the images to allow building Snap Packages in GitHub Actions and Gitlab CI.
I’ve created a PR to fix the securityfs issue with the default WSL2 kernel:
This is now merged and the container images published. You can run docker pull diddledan/snapcraft:core22 (replacing core22 with the tag appropriate for the base that your snap package targets) to get the updated image(s).
The supported tags are:
core (deprecated, due to ESM status of Ubuntu 16.04 - you should be moving off using core as your base snap as soon as possible if your package is still using it)
To be clear I mean the actual Dockerfile, your source, is not for newbies, not the usage of that Docker.
To craft that Dockerfile, it took a lot of insight, and it is not intuitive. This is okay, hiding complexity is perfectly fine, and for someone just using it, it does not really matter. As you point out, the docker run is pretty standard.
It just means that someone (you) will have to have that knowledge to maintain it and keep in touch with the snapcraft team to make sure that new snapcraft features/dependencies do not break the Docker build. Which is why I proposed that the actual snapcraft repo maintainers adopt it. But there could be other ways.
Thanks for letting us know. I think we may have to deprecate this page (at least from the official navigation) as I don’t think there’s currently an official Docker method. I know @lucyllewy has put a lot of effort into maintaining images, but I’ve not run through the process recently.
You’re right that there is currently no official docker support for building snaps. I’ve not rechecked my own images for a couple of months but they were functional and up to date fairly recently. I haven’t had any reports of them failing since I last fixed them, so I think they’re good to go.
