Development sprint March 5th, 2018

Snapcraft pack via snapd

  • Add snap pack --check-skeleton doing validation before pack
  • Do pack in snapcraft via snap pack command
  • snapcraft-snap-tool built with snapd, ships /usr/lib/snapd/snap
  • snapcraft looks for /usr/bin/snap first, then /usr/lib/snapd/snap
  • snapcraft deb depends on (snapcraft-snap-tool || snapd)
  • snapcraft-snap-tool package must be preferred on apt install, so people that don’t have snapd won’t pull it and potentially break their environment (e.g. docker, etc); but /usr/bin/snap must be preferred by snapcraft, so that an updated snapd is used when possible
1 Like

Testing in GCE

  • Spread google backend is ready
  • Pagination bug when listing old images, to be fixed this week
  • Developers have credentials
  • ubuntu-16.04-64 already using it in default snapd tests
  • More images coming soon
  • Login with: gcloud auth application-default login
  • Copy json credentials out of .config/gcloud when using confined spread

Timer services

  • Syntax is the same as refresh syntax (not by coincidence :-))
  • Generate .timer and .service files for systemd
  • snap run --timer only runs command if the window is right
  • Sharp schedules (10:00) allows running within the minute
  • It’s all merged, will be out with snapd 2.33.
1 Like

Allow latest track to be an alias

  • We should add support for a track to be an alias to a different track
  • Track aliases would be transparent to the client
  • Today publishers need to maintain the latest track and the latest major track manually, which is error prone
  • If a track is necessary for a previous major version (say, 1.0), there are good chances that there’s value in having the next major version (say, 2.0) in a track as well, so people can choose to stay in 2.0 even if 3.0 is released

Default track concept

  • Today the default track is the latest, always
  • That’s not very meaningful, as people can push whatever they want on latest
  • Some applications behave poorly when the client migrates from one major to the next
  • Epochs will help that situation, but epochs in general are about migrating the application itself. In some cases, the external users of the application (say, source code for a compiler) will stop working when the application itself is upgraded, even if the upgrade works correctly
  • The concept of a default track would sort that out: if snapd requests stable, unadorned with a track, the server might report back to snapd what the default track is, and that would be set as the current track, with everything else behaving the same from then on

Interface hooks

  • Pretty much ready. Three final PRs need reviews:
    1. Changes for: connect command, auto-connect task, refresh command
    2. Changes for: disconnect hooks, disconnect command, remove command
    3. PolicyCheck changes for Attrer interface
  • Needs test snaps exercising the functionality
  • Tests for auto-connect need a signed snap
  • Plug/Slot types need to be dropped
  • Needs documentation

Refresh deferring command

  • Initial driver for the feature is holding refreshes right after seeding in some cases, but also a generally requested feature
  • snap refresh --hold=2h
  • Effective hold time is the maximum absolute time between current hold and requested hold
  • It does not schedule a refresh after the requested time, but instead prevents the refresh from happening within that window
  • Disabling the hold window is possible via --hold=none
  • Hold time is stored in refresh.hold core configuration setting (public)
  • Hold can change many times before the actual window without affecting it
  • At the refresh attempt, if inside a hold window, skip it as long as it doesn’t go beyond the maximum number of days (60 days right now)
  • Must warn if requested hold cannot be respected due to maximum period
  • In the future, per snap holds will be possible, but probably done via direct question to the snap (hook) instead of via persistent setting
  • First version of the --hold implementation will be optimistic and trust that nobody changed the configuration setting concurrently
  • On a follow up we’ll support configuration “nonces” (or version) that will allow optimistic locking of the configuration: “proceed as long as the config hasn’t changed since I looked”
  • The command work is pending, and recomputing next when inside the hold is also pending, but the underlying implementation is already up for review

Gated xdg-open

  • Provides a way for snaps to open files in the snap filesystem
  • The snap runs xdg-open file://...
  • Request is transferred to the outside world via userd
  • A file descriptor is sent along with the request, and userd checks that the external path refers to the same file
  • A pop-up is opened to let the user decide to open the file or not
  • There’s a 5 minutes timeout to answer the question, after which the request is rejected by default
  • In the future there will be a way to make the choice sticky under some conditions
  • At the moment the popped up window no parent, so shows up in the middle of the screen. In the future we should be able to do better.
  • The implementation of xdg-open uses glib right now. That creates a dependency between snapd (the overall snap) and glib, which is a problem in a multi-base situation. We’ll port it to being a subcommand of snapctl this week.
  • xdg-open becomes a shell script running snapctl user-open file://...

Repairs progress

  • They work for a while now.
  • We have done a staging test, but not a production test yet.
  • Production test to be done in the next two weeks.
  • Future work: early boot runs, auto-import support, brand-signed repairs.

Configuration validation for core

  • Schemas are coming, but will take a little while
  • It’s cheap to just check on configuration changes that the options set are actually supported
  • Apply a patch so that unsupported options already in the configuration are first cleaned up
  • The patch is okay in this case because it does not break backwards compatibility; in fact, it makes it more compatible.
  • This might break people that have been abusing the core configuration for unsupported settings; that’s indeed unsupported and will break.

The “system” nickname

  • Rationale:
    • Gadget configuration defaults are based on snapd IDs
    • The snap IDs are different between staging and production stores, signed by different root keys
    • That makes it hard to write tests for some cases, since the gadget snap needs to refer to both the staging and production core snaps
    • Also, this is the most configured snap by far, so it’s convenient to have a nice short name for it
    • Also, it’s just weird to configure the “core” snap for what is in fact a system setting
    • Interface connections have a similar issue, with core:home being obtuse.
    • These are shorthanded as :home today, but they are still exposed under the core snap.
    • The core snap is becoming a base rather than a singleton, which will confuse things further.
  • For all of those reasons, we’re introducing a “system” label, that can be used in a few places:
    • Configuration options (gadget defaults, snap set/get, etc)
    • Interface connections
  • The “system” snap was registered to ensure it’s not used for other purposes.
  • Future snapd’s must also protect against installing anything named “system”.

Moving core18 build to foundations

  • Goal is to move building of core snaps to foundations team so we can unify security fixes
  • All testing processes currently in place need to be preserved
  • Cannot go directly to candidate without testing processes approving first
  • Foundation snaps will be pushed first on a separate track, then get to candidate and stable
  • Upgrades of core devices need to be figured out separately
  • Any patched packages to be gradually integrated into the archive
  • We might get core16 and core18 moved to foundations first, and then shrink core into irrelevance

So they… what?

Can we make the ubuntu core/base snaps be called something with an ubuntu prefix? The fact that we have only one store where everything lives means that once I get to having a Fedora-based core snap and Fedora base snap, I’d like to be able to configure snapd to use those on Fedora systems…

Sorry, fixed.

The core name is historical and doesn’t seem worth changing now. That said, note that base snaps work a bit differently than what’s suggested there. What decides the base snap to use is the snap itself, not the system it’s running on. That’s critical for the application to continue working correctly across distributions. So if a snap is installed in Fedora, it should continue using whatever core it was built with as a base. Same happens anywhere else: if a snap is built on Fedora with fedora28 (or something) as a base, and the snap is then installed in Ubuntu, snapd will pull fedora28 in and respect its base selection.

wouldnt that be the same on all systems (the “snapd snap” …)

you would only want a fedora-base snap and that you can name as you like …

Entitlements / Powers

  • Work is being deferred as the primary requirement is gone.
  • Design is still looking reasonable.
  • We’ll focus on the full implementation later, instead of rushing it now.

Epochs

  • Works continues on the new store API that is a pre-requisite for epochs
  • Work on epochs specifically to be resumed in about a month.

Parallel installs

  • Will allow snaps to be installed as postgres_prod and postgres_stage, with prod and stage being local keys for the same underlying snap name.
  • There’s no need for any further commands to manipulate such suffixes, every command will work as usual while supporting the adorned name, as in snap install postgres_prod
  • Each installation has completely independent data: version, revision, track, configuration, etc.
  • Commands show the details in context for the given installation, as in snap info postgres_prod showing the exact revision and refresh details for that snap
  • The unadorned name still works, of course, with all existing semantics preserved
  • Bases, kernels, gadgets, will require an unadorned name to be present in the system, and that’s the one that snapd will look into for system-level activities.
  • The store API must get some sort of identifier for each snap (probably a sequence number) so that snapd and the store can communicate about the multiple instances. That identifier is specific to a single API call.
  • The suffix must be constrained so it encourages a small number of chars (probably just [a-z0-9]+), and it must not allow over 10 chars in the first implementation.

Upcoming releases

  • 2.32 was to be released until next Monday
  • Bug in layouts will push out release by a week
  • 2.33 will be difficult to ship in time for Ubuntu 18.04

User mounts

  • We can now create snippets for user mount profiles from interfaces
  • We did not merge the change doing the creation of the user mount namespace
  • Initial phase will have ephemeral namespaces for these cases, which last while the application process is alive.
  • Second phase to follow soon after will persist the user namespaces, and use snap-update-ns to update it dynamically just like everything else today.
  • Current development is addressing a security concern:
    • Unlike all the other mounts which snap-update-ns does, the source and the target of these mounts is owned by the user
    • snap-confine runs as root, so symlinks might trick the system into mounting into the wrong places
    • Unfortunately the kernel API lacks support for operations via file descriptors, and lacks support for a no-follow flag
    • The simplistic idea of doing the mount and then checking is not ideal because it creates a window of exploit, and even if right now the application is frozen during the refresh and the namespace is currently hidden, in the future we want to make that namespace persistent. Also the principle of it is in the wrong side of history: do-and-check is bad security vs. don’t do anything dangerous in the first place.
    • Current plan is to do the mount with a target in a non-accessible location, then verifying, and just then exposing to the world.
    • One way to verify might be to open the directory, inspect the path associated with the file descriptor to ensure correctness, doing the bind mount, and then comparing that the inode is the same, and finally exposing the target location to the application
    • The way to expose securely at the final step is to go to the target directory and mount from within after ensuring correctness
  • Further investigation will take place next week to ensure correctness and safety.
  • Landing of working final ultra safe version (maybe broken) is after security review takes place, so probably two weeks from now.

Portals

  • Portals work by allowing confined applications to send messages to the portal
  • Portals check the identity of the call site and make a decision
  • Support for snaps was merged upstream but it was malfunctioning
  • Support for snaps was fixed upstream already, but not released yet
  • Ubuntu might be able to support the fixed version soon, and Fedora likely has a fast paced schedule on that too; need to coordinate with Debian and others?
  • How to check if there’s a working version?
  • Even then, fixed version will take a while to propagate to every place, and will never be seen in some old releases, which makes snaps using that feature incompatible with those. That’s a side effect of API evolution, but we need to understand how best present the situation.
  • There’s a security concern that needs further discussion: if portals think the snap is actually unconfined, they might fail open. Needs further investigation and impact analysis.
1 Like

If I’m making something that mimics Ubuntu Core, I’d need to make a custom central snap. Right now, that’s core.

Also, Fedora’s snapd is already different from yours, so if I relied on the one Ubuntu builds, it’d fail.

Yeah, bases already allow you to have a custom core for Fedora, and after we’re done with the things we want to get into 18.04, we’ll come back to finish the multi-base support in Ubuntu Core. The change that will enable us to have Ubuntu Core 18 is the same that will enable you to have a Fedora Core.

1 Like

As soon as snaps can use portals, let me know which ones and what releases need to be available so that I can work with the Fedora maintainers of those packages to get those integrations in place across as many supported releases as possible.

Currently, I can guarantee it being available in Fedora 28 (which will release this May) if Alex Larsson’s PR is merged soon and a release is made, but bringing it back to Fedora 26 and Fedora 27 will require some cooperation, and I’ll probably only entertain those discussions once snapd actually does support portals.

2 Likes

Thanks, Neal! Let’s catch up soon once we have something working.