Call for testing docker snap

[gary-wzl]

Hi guys
We’re close to publishing the docker snap into stable channel. Before that, we’d like to ask you guys to help test the snap.
1. To install it, please run the following command.

    $ snap install --candidate docker

After installation, a little manual setup is required to overcome the confinement. The setup is a little bit different though on classic and Ubuntu Core 16. Please run the following command to get it.

   $ docker.help

Currently, the snap on candidate channel is built against docker 1.13
We also plan to follow the new release cycle that the way Docker Inc run for their upcoming releases and publish the first quarterly release of docker snap(17.03) later in a new track.
Thanks.

BR
Gary.Wzl

[lazypower]

Greetings Gary,

I’ve been using the docker snap since 1.11 was stuffed in a .snap and the experience has been fairly pleasant.

I’m curious about a few things though:

• Will we be making a Moby snap as well or continue basing the snaps off the downstream docker release? (I suspect the latter, as Moby is upstream and component based)
• What will the channel format be? (As a server-side snap, this is an important detail for me as the juju layer-docker maintainer)
• Are we going to be snapping up both DockerCE and DockerEE? Will there be major differences between the two?

I’ve avoided adding snap support to layer-docker thus far due to the non-beta label on the work, and the fact that auto refreshing snaps on a server means restarting the workloads (potential downtime). However as a developer-machine installation of the docker-snap has been fantastic thus far without any issues from my end. I’ve been using it as a daily driver for quite some time now.

Also, in closing, great work on the snap here so far. It’s been pleasant to get updates automatically without any interaction on my part.

[danilo]

У пет, 21. 04 2017. у 14:27 +0000, Charles Butler пише:

I’ve avoided adding snap support to layer-docker thus far due to the
non-beta label on the work, and the fact that auto refreshing snaps
on a server means restarting the workloads (potential downtime).

Even manual snap installation will restart the workloads:

https://bugs.launchpad.net/snapd/+bug/1664163

I suppose you’d want that to be optional, just like we do: I am
guessing it would be a common thing when pairing juju charms with
snaps, because charms would control the workload status, and snaps
would be a delivery mechanism.

Changing it in snapd wouldn’t be too hard, but it’s more about agreeing
that this is desireable.

Cheers,
Danilo

[gary-wzl]

Thanks for your feedback.

• Will we be making a Moby snap as well or continue basing the snaps off the downstream docker release? (I suspect the latter, as Moby is upstream and component based)
• What will the channel format be? (As a server-side snap, this is an important detail for me as the juju layer-docker maintainer)
• Are we going to be snapping up both DockerCE and DockerEE? Will there be major differences between the two?

1.Yes, we adopted the latter one. Basically, as the Docker Engine is split up into multiple components,e.g runc, containerd, libnetwork and might be more in the future.
Currently, the way we snapped the docker is based on downstream docker release,
snap each component respectively and bundle all components into one single package.
That makes us life easier to add additional component into the snap sooner or later.
docker-snap snapcraft.yaml file

2.I guess you mean “the track format.”
In general, we’ll follow a YY.MM versioning scheme that Docker Inc used. The track name for first quarterly release will be 17.03-ce.
We’ve been discussing this on the store category.
Docker snap tracks reqeust

3.I’d like to discuss with team. So far, there’s no plan to support both.
The major differences for those two come down to upgrade policy, lifecycle and maintenance cycle. There’s an official blog entry about a comparison of both.
Announcing Docker Enterprise Edition - Docker Blog

[lazypower]

I certainly think we are on the same page here. I suppose its fine to indicate that upgrading the charm (thus upgrading the snap package) is going to restart your workload, and indicate that there is now a clear coupling between these events in most cases.

Additionally, once the docker daemon auto snap-refreshes to catch that latest bug-fix in the channel, its going to interrupt workload. This will work in some cases, but I think we need to look at the big picture for this use case which is largely different than just a developer installation.

For example, if you’re running a web-store at the scale of the Ubuntu Store, you’d have scaled your workers and any snap refreshes you incur (so long as they were staggered) would be load balanced and you can feesibly lose one or more nodes and that failover routing would ensure your customers aren’t impacted.

However, if they aren’t staggered, you’re going to feel that bite when your container(s) restart. That’s my primary concern is ensuring the charm retains predictable behavior, and the consumer doesn’t get bit by operational pain. I’ll see about adding this feature in the current cycle and making it a toggle option during install that defaults to true during the edge => stable release phase so we can get early adopter feedback.

This may prevent any need to change anything in snapd, but i suspect that over time we’ll shake out the questions i’m failing to come up with today.

[zyga]

I just want to encourage everyone to give this a try on Fedora and Debian.
On OpenSUSE it will fail as we’re aware of an issue with how the snap execution environment is set up. We’re aware of the issue and have started fixing it but it will be 2.27 maybe 2.28 when this is addressed.

[lazypower]

I’ve syndicated to my social feeds on this matter. <3 encourage you all to do the same. Lets help get some exposure on this open call for testing on distro’s other than Ubuntu. I know that 95% of my personal testing has been on Ubuntu, with some brief layovers in Debian Sid.

[gary-wzl]

Hey @zyga,
Today I spent some time testing the docker-snap on Fedora. In general, after some tweaks, docker-snap works fine in snappy world on Fedora, but there’s one thing I noticed during the verification.
Currently, it seems to me people have to disable SELinux to make snap work on Fedora, otherwise docker command hangs due to SELinux avc denies.
Is there any plan to run a snap without SELinux disabling?
Thanks.

BR
Gary

[gary-wzl]

Hi guys
The docker snap 17.03.1-ce-0 is out now. You can install it with the following command

$ snap install --candidate docker

In this release,
1.we removed the account-control plug in the snap for avoiding it being abused. Another benefit of this is that for the sake of easy setup, docker daemon is up and running after the installation without account-control connected and snap reloaded.
2.we included tini package now for the sub-command of docker(docker run --init)

If everything goes smoothly, we’ll promote it to stable channel shortly.
Thanks.

BR
Gary

[gary-wzl]

Hi guys
We’ll release the 17.03.1-ce-1 (revno 118) to stable channel soon. Before that, you can have a try from candidate channel.

 $ snap install --candidate docker

The major feature in this release is that deploying a secure private registry with docker snap is possible now. The certs folder now is placed into a writable path (os.Getenv(“SNAP_COMMON”) + “/etc/certs.d”) within sandbox so that docker client can access for authentication.

Meanwhile, you can check this out about how to setup a private registry server.
In the near future, a configurable option for certs folder path would be supported via hooks once a new CLI parameter is introduced officially by docker.

Thanks

[ogra]

when testing with https://github.com/ogra1/snapd-docker which starts a detached container and leaves it running, i noticed that the container gets stopped but not restarted after the refresh from stable to candidate (or back from candidate to stable)…

it would be nice if the snap could actually record the state of running detached containers and start them again after refreshes.

[gary-wzl]

Thanks for your testing and suggestion.
We’ll take this into account and include this feature in our next release if it looks good to docker Inc too.

[ogra]

i’m not sure if this is actually much relevant for docker inc.

if you have a deb or rpm, that will most likely not quietly auto-refresh without an admin noticing … snaps do that though so if a quiet background refresh happens because there is a new docker snap in the store, your detached running containers will all end up stopped in the middle of your workday …

i think this is something that needs to be handled specifically for snaps (though if docker inc. thinks it is generally useful on all kind of package upgrades, we could surely benefit from such a feature, but snaps are a bit more radical with their update approach)

[gary-wzl]

Basically, there’s a similar discussion around this based on the customer feedbacks.

When UC16 updates and autonomously reboots - all of the containers stalled with no notice. Ideally - it would be nice if a dockerized solution using multiple microservices/containers had a framework or system on UC16 that would allow for autonomous reboots, and do something smartly to ensure containers stay continuously running.

It’s in our backlog. As now docker Inc owns the snap again, from their opinion, snap package should adapt docker to snap, but it should not go beyond what docker on other platforms does.
An intermediate step, 1) launch a container with a restart policy applied 2) add “restart: always” to services in the docker-compose.yaml file will automatically restart services. That keeps the same behavior on all platforms.
We still need to communicate with docker Inc somehow at this point and hear their opinions.

Thanks.

[psftw]

Hello. I’m on the Docker, Inc side of the equation and have been getting up to speed on the Docker snap packaging recently. In this case, I think Gary summarized our philosophy well, and I don’t see the need for any snap-specific changes.

[gary-wzl]

Hi guys
As you may know, Docker, Inc just released 17.06.0-ce and 2nd patch version of 17.03-ce two days ago. And docker snap regular releases are driven by upstream, we followed up their footsteps and released snaps to candidate channel of respective track.

• 17.03.2-ce-1 (track: latest and 17.03)
  Update to docker-ce 17.03.2-ce and compose utility to 1.14.0.

• 17.06.0-ce-1 (track: 17.06)
  Update to docker-ce 17.06 and compose utility to 1.14.0.

We also included docker machine tool in each snap package and added content-interface slot in each release.
With content-interface, it enables developers to create a dockerized app snap and use it as delivery mechanism to leverage docker and its family utilities within its own sandbox. Here is a simple experimental snap that use content-interface shared by docker snap to invoke docker command.

Meanwhile, according to our release planning,

• For users who stay on the latest track, they will get auto-upgraded to 17.03.2-ce-1 once we promote it to stable channel and then later 17.06.0-ce-1 at some point in ~1 month,

• For users who pick to stay on 17.03 or 17.06 today, that will require their manual attention to switch tracks in the future when 17.03 or 17.06 release comes to an end.

We’d recommend people who want to keep getting quarterly release and its patch version staying on the latest track.

Please give it a try and feedback is appreciated.

Thanks.

[jdstrand]

We also included docker machine tool in each snap package and added content-interface slot in each release.
With content-interface, it enables developers to create a dockerized app snap and use it as delivery mechanism to leverage docker and its family utilities within its own sandbox. Here is a simple experimental snap that use content-interface shared by docker snap to invoke docker command.

I took a look at your docker-app snap and wanted to mention that it is a novel approach to have the docker snap export a content interface that has the tools that an app snap could use to drive docker for application containers delivered as snaps. I do want to mention that in looking at the docker app snap, you are having to plugs both docker-support and docker which means:

• because of docker-support, the app snap will need a snap declaration to install from the store (because the docker-support interface is super-privileged)
• because of docker, the app snap will need to have the interface connected manually

Because of docker’s capabilities (docker-support is meant for dockerd which requires extensive permissions to run) and because of docker’s design (access to the dockerd socket gives full access to all of docker’s capabilities), both interfaces allow trivial confinement breakout and are intended only for use with trusted snaps from trusted publishers since when plugging either interface, you have in practice granted the app snap device ownership. As such, I strongly advise against using this mechanism with untrusted docker app snaps. For trusted snaps for a brand store, this may be interesting (since brand stores can issue snap declarations allowing these connections). For the public store, developers will be frustrated trying to deliver application containers in this manner since they’ll bump into the base declaration’s policy.

If delivering application containers via snaps in a general-purpose, developer-friendly and secure manner in the public store is important, I suggest working with docker upstream (eg, using a proxy, putting ACLs on the docker socket APIs, introducing another socket with secure APIs, etc).

[gary-wzl]

Hi everyone
For various reasons, docker-snap lagged behind the official docker-ce release and we’re doing our best to fill the gap in docker snap release.

• Today we published 17.06.1-ce docker snap in the store. You can find it at the stable channel of latest or 17.06 track.

• The latest patch version of 17.06 docker snap (17.06.2-ce) has been promoted into the candidate channel. The changelog for this version can be found here

People who are still staying at 17.03 track won’t have automatic updates for this release(17.06.1-ce) since the snap won’t get upgrades across tracks. So for those who are staying at 17.03 track would like to try the latest features of docker snap in 17.06, you need to manually uninstall the previous version and install the docker snap from the latest or 17.06 track from the command line

$sudo snap install docker

or

$sudo snap install --channel=17.06/stable docker

Your feedback is highly appreciated.
Thanks.

[gary-wzl]

Hi guys
We just published two versions of docker snap into the store today.

• 17.06.2-ce is promoted to the stable channel of “latest” and “17.06” tracks. It will be the last revision of 17.06 ce release.

• The latest revision of 17.09 ce quarterly release(17.09.1-ce) has been promoted to the candidate channel of the newly created track “17.09”. The changelog for this revision can be found here

As usual, for those who are staying at “17.03” or “17.06” track would like to try the latest features of docker snap against 17.09.1-ce, you need to manually uninstall the previous version and install the docker snap from the “17.09” track from the command line

$sudo snap install --channel=17.09/candidate docker

We plan to promote 17.09.1-ce to the stable channel of “17.09” track in a couple of weeks if we receive positive feedback from the community and one month later after its general availability, we will release 17.09.1-ce to the stable channel of “latest” track so that users of “latest” track will be automatically upgraded to 17.09 without making any changes

Thanks.

[daniel]

With btrfs root filesystem I’m failing to get docker running at all. With 17.06 the daemon starts but trying to run hello-world results in:

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Extracting     985B/985B
docker: failed to register layer: ApplyLayer exit status 1 stdout:  stderr: mkdir /temp-docker-extract636676059: permission denied.
See 'docker run --help'.

Similar messages occur with any docker pull command.

dmesg reports:

$ dmesg |tail -n1
[167212.136046] audit: type=1400 audit(1514766156.307:2091): apparmor="DENIED" operation="mkdir" profile="snap.docker.dockerd" name="/temp-docker-extract636676059/" pid=32725 comm="exe" requested_mask="c" denied_mask="c" fsuid=0 ouid=0

With 17.09/candidate I cannot get the daemon to start at all, so can’t check this issue in the later docker release.