Call for testing docker snap

У пет, 21. 04 2017. у 14:27 +0000, Charles Butler пише:

I’ve avoided adding snap support to layer-docker thus far due to the
non-beta label on the work, and the fact that auto refreshing snaps
on a server means restarting the workloads (potential downtime).

Even manual snap installation will restart the workloads:

I suppose you’d want that to be optional, just like we do: I am
guessing it would be a common thing when pairing juju charms with
snaps, because charms would control the workload status, and snaps
would be a delivery mechanism.

Changing it in snapd wouldn’t be too hard, but it’s more about agreeing
that this is desireable.


Thanks for your feedback.

1.Yes, we adopted the latter one. Basically, as the Docker Engine is split up into multiple components,e.g runc, containerd, libnetwork and might be more in the future. Currently, the way we snapped the docker is based on downstream docker release, snap each component respectively and bundle all components into one single package. That makes us life easier to add additional component into the snap sooner or later. docker-snap snapcraft.yaml file

2.I guess you mean “the track format.” :slight_smile: In general, we’ll follow a YY.MM versioning scheme that Docker Inc used. The track name for first quarterly release will be 17.03-ce. We’ve been discussing this on the store category. Docker snap tracks reqeust

3.I’d like to discuss with team. So far, there’s no plan to support both. The major differences for those two come down to upgrade policy, lifecycle and maintenance cycle. There’s an official blog entry about a comparison of both. Announcing Docker Enterprise Edition - Docker Blog

1 Like

I certainly think we are on the same page here. I suppose its fine to indicate that upgrading the charm (thus upgrading the snap package) is going to restart your workload, and indicate that there is now a clear coupling between these events in most cases.

Additionally, once the docker daemon auto snap-refreshes to catch that latest bug-fix in the channel, its going to interrupt workload. This will work in some cases, but I think we need to look at the big picture for this use case which is largely different than just a developer installation.

For example, if you’re running a web-store at the scale of the Ubuntu Store, you’d have scaled your workers and any snap refreshes you incur (so long as they were staggered) would be load balanced and you can feesibly lose one or more nodes and that failover routing would ensure your customers aren’t impacted.

However, if they aren’t staggered, you’re going to feel that bite when your container(s) restart. That’s my primary concern is ensuring the charm retains predictable behavior, and the consumer doesn’t get bit by operational pain. I’ll see about adding this feature in the current cycle and making it a toggle option during install that defaults to true during the edge => stable release phase so we can get early adopter feedback.

This may prevent any need to change anything in snapd, but i suspect that over time we’ll shake out the questions i’m failing to come up with today.

I just want to encourage everyone to give this a try on Fedora and Debian.
On OpenSUSE it will fail as we’re aware of an issue with how the snap execution environment is set up. We’re aware of the issue and have started fixing it but it will be 2.27 maybe 2.28 when this is addressed.


I’ve syndicated to my social feeds on this matter. <3 encourage you all to do the same. Lets help get some exposure on this open call for testing on distro’s other than Ubuntu. I know that 95% of my personal testing has been on Ubuntu, with some brief layovers in Debian Sid.

Hey @zyga-snapd,
Today I spent some time testing the docker-snap on Fedora. In general, after some tweaks, docker-snap works fine in snappy world on Fedora, but there’s one thing I noticed during the verification.
Currently, it seems to me people have to disable SELinux to make snap work on Fedora, otherwise docker command hangs due to SELinux avc denies.
Is there any plan to run a snap without SELinux disabling?


Hi guys
The docker snap 17.03.1-ce-0 is out now. You can install it with the following command

$ snap install --candidate docker

In this release,
1.we removed the account-control plug in the snap for avoiding it being abused. Another benefit of this is that for the sake of easy setup, docker daemon is up and running after the installation without account-control connected and snap reloaded.
2.we included tini package now for the sub-command of docker(docker run --init)

If everything goes smoothly, we’ll promote it to stable channel shortly.


Hi guys
We’ll release the 17.03.1-ce-1 (revno 118) to stable channel soon. Before that, you can have a try from candidate channel.

 $ snap install --candidate docker

The major feature in this release is that deploying a secure private registry with docker snap is possible now. The certs folder now is placed into a writable path (os.Getenv(“SNAP_COMMON”) + “/etc/certs.d”) within sandbox so that docker client can access for authentication.

Meanwhile, you can check this out about how to setup a private registry server.
In the near future, a configurable option for certs folder path would be supported via hooks once a new CLI parameter is introduced officially by docker.


when testing with which starts a detached container and leaves it running, i noticed that the container gets stopped but not restarted after the refresh from stable to candidate (or back from candidate to stable)…

it would be nice if the snap could actually record the state of running detached containers and start them again after refreshes.

Thanks for your testing and suggestion.
We’ll take this into account and include this feature in our next release if it looks good to docker Inc too.

i’m not sure if this is actually much relevant for docker inc.

if you have a deb or rpm, that will most likely not quietly auto-refresh without an admin noticing … snaps do that though so if a quiet background refresh happens because there is a new docker snap in the store, your detached running containers will all end up stopped in the middle of your workday …

i think this is something that needs to be handled specifically for snaps (though if docker inc. thinks it is generally useful on all kind of package upgrades, we could surely benefit from such a feature, but snaps are a bit more radical with their update approach)

Basically, there’s a similar discussion around this based on the customer feedbacks.

When UC16 updates and autonomously reboots - all of the containers stalled with no notice. Ideally - it would be nice if a dockerized solution using multiple microservices/containers had a framework or system on UC16 that would allow for autonomous reboots, and do something smartly to ensure containers stay continuously running.

It’s in our backlog. As now docker Inc owns the snap again, from their opinion, snap package should adapt docker to snap, but it should not go beyond what docker on other platforms does.
An intermediate step, 1) launch a container with a restart policy applied 2) add “restart: always” to services in the docker-compose.yaml file will automatically restart services. That keeps the same behavior on all platforms.
We still need to communicate with docker Inc somehow at this point and hear their opinions.


1 Like

Hello. I’m on the Docker, Inc side of the equation and have been getting up to speed on the Docker snap packaging recently. In this case, I think Gary summarized our philosophy well, and I don’t see the need for any snap-specific changes.


Hi guys
As you may know, Docker, Inc just released 17.06.0-ce and 2nd patch version of 17.03-ce two days ago. And docker snap regular releases are driven by upstream, we followed up their footsteps and released snaps to candidate channel of respective track.

  • 17.03.2-ce-1 (track: latest and 17.03)
    Update to docker-ce 17.03.2-ce and compose utility to 1.14.0.

  • 17.06.0-ce-1 (track: 17.06)
    Update to docker-ce 17.06 and compose utility to 1.14.0.

We also included docker machine tool in each snap package and added content-interface slot in each release.
With content-interface, it enables developers to create a dockerized app snap and use it as delivery mechanism to leverage docker and its family utilities within its own sandbox. Here is a simple experimental snap that use content-interface shared by docker snap to invoke docker command.

Meanwhile, according to our release planning,

  • For users who stay on the latest track, they will get auto-upgraded to 17.03.2-ce-1 once we promote it to stable channel and then later 17.06.0-ce-1 at some point in ~1 month,

  • For users who pick to stay on 17.03 or 17.06 today, that will require their manual attention to switch tracks in the future when 17.03 or 17.06 release comes to an end.

We’d recommend people who want to keep getting quarterly release and its patch version staying on the latest track.

Please give it a try and feedback is appreciated.


1 Like

I took a look at your docker-app snap and wanted to mention that it is a novel approach to have the docker snap export a content interface that has the tools that an app snap could use to drive docker for application containers delivered as snaps. I do want to mention that in looking at the docker app snap, you are having to plugs both docker-support and docker which means:

  • because of docker-support, the app snap will need a snap declaration to install from the store (because the docker-support interface is super-privileged)
  • because of docker, the app snap will need to have the interface connected manually

Because of docker’s capabilities (docker-support is meant for dockerd which requires extensive permissions to run) and because of docker’s design (access to the dockerd socket gives full access to all of docker’s capabilities), both interfaces allow trivial confinement breakout and are intended only for use with trusted snaps from trusted publishers since when plugging either interface, you have in practice granted the app snap device ownership. As such, I strongly advise against using this mechanism with untrusted docker app snaps. For trusted snaps for a brand store, this may be interesting (since brand stores can issue snap declarations allowing these connections). For the public store, developers will be frustrated trying to deliver application containers in this manner since they’ll bump into the base declaration’s policy.

If delivering application containers via snaps in a general-purpose, developer-friendly and secure manner in the public store is important, I suggest working with docker upstream (eg, using a proxy, putting ACLs on the docker socket APIs, introducing another socket with secure APIs, etc).


Hi everyone
For various reasons, docker-snap lagged behind the official docker-ce release and we’re doing our best to fill the gap in docker snap release.

  • Today we published 17.06.1-ce docker snap in the store. You can find it at the stable channel of latest or 17.06 track.

  • The latest patch version of 17.06 docker snap (17.06.2-ce) has been promoted into the candidate channel. The changelog for this version can be found here

People who are still staying at 17.03 track won’t have automatic updates for this release(17.06.1-ce) since the snap won’t get upgrades across tracks. So for those who are staying at 17.03 track would like to try the latest features of docker snap in 17.06, you need to manually uninstall the previous version and install the docker snap from the latest or 17.06 track from the command line

$sudo snap install docker


$sudo snap install --channel=17.06/stable docker

Your feedback is highly appreciated.

Hi guys
We just published two versions of docker snap into the store today.

  • 17.06.2-ce is promoted to the stable channel of “latest” and “17.06” tracks. It will be the last revision of 17.06 ce release.

  • The latest revision of 17.09 ce quarterly release(17.09.1-ce) has been promoted to the candidate channel of the newly created track “17.09”. The changelog for this revision can be found here

As usual, for those who are staying at “17.03” or “17.06” track would like to try the latest features of docker snap against 17.09.1-ce, you need to manually uninstall the previous version and install the docker snap from the “17.09” track from the command line

$sudo snap install --channel=17.09/candidate docker

We plan to promote 17.09.1-ce to the stable channel of “17.09” track in a couple of weeks if we receive positive feedback from the community and one month later after its general availability, we will release 17.09.1-ce to the stable channel of “latest” track so that users of “latest” track will be automatically upgraded to 17.09 without making any changes


1 Like

With btrfs root filesystem I’m failing to get docker running at all. With 17.06 the daemon starts but trying to run hello-world results in:

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Extracting     985B/985B
docker: failed to register layer: ApplyLayer exit status 1 stdout:  stderr: mkdir /temp-docker-extract636676059: permission denied.
See 'docker run --help'.

Similar messages occur with any docker pull command.

dmesg reports:

$ dmesg |tail -n1
[167212.136046] audit: type=1400 audit(1514766156.307:2091): apparmor="DENIED" operation="mkdir" profile="snap.docker.dockerd" name="/temp-docker-extract636676059/" pid=32725 comm="exe" requested_mask="c" denied_mask="c" fsuid=0 ouid=0

With 17.09/candidate I cannot get the daemon to start at all, so can’t check this issue in the later docker release.

Thanks for your reporting.
I hit this problem too when I changed the default storage driver from aufs -> btrfs.

Jan 2 14:42:41 VBox kernel: [10586.144283] audit: type=1400 audit(1514875361.547:224): apparmor=“DENIED” operation=“mkdir” profile=“snap.docker.dockerd” name="/temp-docker-extract625899634/" pid=10465 comm=“exe” requested_mask=“c” denied_mask=“c” fsuid=0 ouid=0
Jan 2 14:42:41 VBox docker.dockerd[9657]: time=“2018-01-02T14:42:41.624544086+08:00” level=debug msg=“Cleaning up layer afc87d762f6722ef3b8610900570ebb7d08cb97a193d62a30116d18185d7b86d: ApplyLayer exit status 1 stdout: stderr: mkdir /temp-docker-extract625899634: permission denied”

For now, we only have a few storage backends(aufs, overlay, zfs) supported in docker snap to make sure it works well both on core and classic. I’ll investigate it and put any updates in this thread.

I take a closer look at this problem.
I am not sure if pivot_root works as expected in snappy. But it looks to me that daemon actually still lives in old root even after pivot_root, which causes apparmor denial(make a dir under ‘/’) during applying the layer.

Probably, it was a duplicate or something related to the following issues

These issues not only exist in overlayfs, but also in btrfs. Thanks @jdstrand for the example code for the bug reproduce.

A quick workaround is using the realChroot directly in snappy. Using the realChroot is the default method for docker daemon to change root in unix and fallbacks in linux once an error occurs during pivot_root
A newer version of 17.06(17.06.2-ce-1 rev: 237) has been uploaded to edge channel of latest/17.06.
@lucyllewy could you help to test it under your environment?
Here is how I tested 17.06.2-ce-1 on classic(16.04) with btrfs storage backend enabled.

A slight difference from your environment, I created a dedicated block device as a btrfs filesystem and manually mounted it to docker snap root dir: /var/snap/docker/common/var-lib-docker