Introducing base snaps

upcoming
mvo

#1

We would like to add a new kind of snap: base.

The core ideas around base are the following:

  • the current “core” shrinks to just contain snapd and its basic tools. We need to figure out what that means for our Ubuntu Core images (all-snaps).
  • we add new base snap type that contains the “base” filesystem for applications to run in. Some possible examples for the base could be e.g. ubuntu-16.04, fedora-25, debian-9
  • the current “core” snap is made available as “ubuntu-16.04”
  • a snap can declare what base it needs, we need to come up with some nice yaml for this. If nothing is declared we default to ubuntu-16.04 for backward compatibility.
  • when snapd detects that a base snap is missing its automatically downloaded and installed together with the snap that needs it. We should also garbage collect it if there is no snap needing it anymore.

This should make snapping software that is written with for specific distros much simpler.

Open questions:

  • how do apps get tools like snapctl inside their execution environment ?
  • does this change anything for re-exec?

Implementation plan (strawman)

Step 1 (groundwork)

  • add new type: base to snap.yaml so that a snap can declare it is a base-snap
  • add new base: some-base-name so that a snap can declare it needs a specific base-snap
  • ensure that a snap that needs a base that is not installed gets rejected (we will add auto-install later)
  • modify snap-confine so that it DTRT for snaps that need base snaps:
  • instead of mounting “/” (ubuntu-core) or “/snap/core/current” (classic) it needs to check a new SNAP_CONFINE_BASE_SNAP environment (or a different way to communicate this information from snapd to snap-confine)
  • mount /snap/$SNAP_CONFINE_BASE_SNAP/current as “/”
  • bind mount /dev, /proc/, /sys, /tmp, /home, /etc(?), /media(?) from the host into the snap “/”

Started with that here: https://github.com/snapcore/snapd/compare/master...mvo5:f/base-snaps?expand=1

Step 2 (transition for classic)

  • ensure snap install/refresh auto-fetches missing base snaps
  • create “ubuntu-16.04” base snap with just the libs/essentials from the current core snap (OPEN QUESTION: what about snapctl - would that mounted in from the core snap?)
  • change snapd so that if no base is specified the “ubuntu-16.04” base is automatically fetched/used

Step 3 (transition on ubuntu-core system)

  • add new ubuntu-core-16 os snap that looks like the current “core”
  • add code to snapd that will fetch ubuntu-core-16 automatically (transition from core to ubuntu-core-16 on all-snap devices)
  • shrink down “core” so that it really just contains the bare essentials to run snapd itself

Syntax for build-snaps
Slightly harsh description of snappy on the AppImage wiki
Weeks 24-29 of 2017 in snapd
The snapd roadmap
The snapd roadmap
Base Snaps - Doubts
Python-gi and Python 2 in a snap of Pick
Python-gi and Python 2 in a snap of Pick
Inspecting package differences across core/base snaps
Snapcraft top-level metadata
Process for reviewing base snaps
Ubuntu core 18?
#2

Do you have a feeling of how base snaps fit into the concept of having a “core” snap that supports far more than needed otherwise just to be bootable in an all-snap system?

Do you envision a thin ubuntu-16 base snap that is used for running apps alongside a big core snap that has systemd and a whole lot of other stuff that is only used because it’s also used to boot a core system?

EDIT:

What I’m trying to get at is this:

core system:

  • core snap
  • kernel snap
  • gadget snap
  • any number of base snaps and app snaps

classic system:

  • core snap
  • any number of base snaps and app snaps

Can we skip the core snap in a classic distro (or most of the content of the core snap, except for snapd)


#3

The current proposal as it stands is that core remains as the required pieces to run snapd proper on top of any base, so core remains necessary both on pure snap devices and on classic systems. Then, pure snap devices would likely get a new base snap defined in the model which specifies the root filesystem, so the foundation becomes core+base+kernel+gadget. Then, when installing a new snap its base would be considered as usual in classic.


#4

So are you saying that say, all-snap pi3 would get a ubuntu-core-16 base snap to assist in booting that would not be used by any app snap at runtime?


#5

We need to discuss this in detail, my current feeling is that for a classic system it will look just like you outline above: a single core snap with snapd (and potentially systemd, base, coreutils, see below), any number of base snaps and app snaps.

For ubuntu core device and core we have two options:

  1. core contains systemd,bash,coreutils(or busybox), snapd, just the bare minimum to boot a ubuntu core device. the application runtime is provided via ubuntu-16.04
  2. core contains just snapd, the boot environment if provided via the ubuntu-core-16 snap

I lean towards option (2) because it means we can make “core” even smaller, but we should explore this a bit more before making a decision.


#6

Yes this would lean to having a ubuntu-core-16 (or even just ubuntu-core as I don’t think this will be ever versioned like -18 is, otherwise we are back to another flavour of the same problem). This way it is clear what is needed in each base snap and that base snaps be in fact smaller to the point of being automatically managed by snapd as transparently pulled in dependencies akin to what you wrote in the initial proposal.


#7

Is there other discussion or a design anywhere else or is this still the latest discussion on base snaps?


#8

@jhodapp We discussed it quite a bit during the sprint in London, and there’s a whiteboard picture that resulted from these discussions in the sprint topic, but it may be a bit hard to grasp what it says without being part of the convershation.

The basic ideas around base snaps should be pretty straightforward, though. Most of the discussions revolved around the inner details of how we plan to put these in place.

I’d be happy to respond to specific questions about the feature, if you have any. We’ll also be producing more documentation soon.


#9

I’d be happy to respond to specific questions about the feature, if you have any. We’ll also be producing more documentation soon.

Thanks @niemeyer, no specific questions quite yet other than the one you just answered. I’d be happy to help review any documentation around this topic and it’d be a good way of making sure someone new to the topic understands the concept. Looking forward to seeing more on it.


#10

Can I ask: What is the ‘Base Snaps’ in ‘upcoming’?


#11

You could try reading the topic that’s linked next to it :stuck_out_tongue: (which Niemeyer has now moved our comments to!)
As I understand it, though, base snaps is a concept where rather than all snaps having to rely on the core snap + whatever platform snaps it’s using as plugs (if any) + stage-packages (and build-packages), they can rely on a minimal core snap + a base snap + plugs (if any) + stage-packages (and build-packages). At the moment the stuff in core is based on Ubuntu release (Ubuntu 16.04 currently, I think), whereas base snaps can be based on any distro and the application snap simply specifies which base snap they want when they’re installed.


#12

Hey, is there any (rough) ETA for base snaps?


#13

Maybe someone actually working on it will respond(!) but it’s not scheduled for snapd 2.29 so presumably late November at the earliest?


#14

Hey @snapd team, this one needs an answer from somebody in the know. :slight_smile:


#15

@TimSueberkrueb Thanks for your question and sorry for my slow reply. We are currently actively working on base snaps. The first basic support is part of the upcoming 2.28 release which is already in the beta channel. It will support bases that look similar to the current “core”, i.e. bases that are (roughly) ubuntu/debian based. As a nice feature it also supports the “bare” base which is an empty base snap. An example snap that uses this empty base is the “busybox-static” snap.

The next version 2.29 will contain support for automatic download of missing bases and fixes for bases that use a fedora/arch layout.

Please let me know if this fits your use case!


#16

@mvo thanks a lot for the detailed answer! Yeah, I’m really interested in an Arch based base snap :slightly_smiling_face:
Looking very much forward to it, keep up the great work!


#17

Is this detected by build.snapcraft.io to choose the builder to use?