Multiple users and groups in snaps

[jdstrand]

The previous (obsolete) discussion on the topic is here. To make the discussion clearer I was asked to start a new topic with the currently agreed to design so that we may work through any additional details. As such, I will copy the entire ‘Introduction’ and a description of the 4 use cases from the other topic, but leave the ‘Implementation Ideas’ in the other topic for historical reference.

Introduction

Today, snappy’s notion of users (and groups) is primitive:

• daemons run as root
• commands run as the user invoking the command

For an initial implementation this has worked very well. The (untrusted) snaps run under root-strong confinement and the access to various resources is mediated via security policy.

There are quite a few important use cases that are not covered by the current implementation however:

1. adding system users/groups https://launchpad.net/bugs/1606510
2. supporting device access via ACLs for granting access of devices to (non-root) users. https://launchpad.net/bugs/1646144
3. per-snap opt-in users to support things such as privilege dropping/separation within snaps. https://launchpad.net/bugs/1619888
4. Supporting uids/gids in chroot environment

‘1’ and ‘2’ deal with users accessing resources that have been granted to the snap via the snap’s interface security policy and highlight the fact that while the snap may have access to a resource, that doesn’t necessarily mean that the user of the snap should have access to that resource.

‘3’ and ‘4’ deal with supporting a wider range of existing software.

This is a complicated topic and while all 4 cases don’t necessarily need to be implemented at the same time, they all should be considered when designing how users and groups are supposed to work in a snappy world. This topic is intended to help people assigned to implementing any or all of these cases to have the necessary design.

Let’s first consider each use case in a little more detail.

System user and groups

Existing software often uses the concept of users and groups as a security mechanism to grant access to some application resource via traditional users and groups. Examples include lxd, docker and libvirt which all create UNIX sockets with 0660 permissions with ownership of root:<some group>. They all do this because the socket provides privileged access to the host system that should only be granted to trusted users. In order for a non-root user to be able to use these services, the user must be added to the system group for the service.

Today, users that want to use snaps that use this security mechanism must either:

• always run the command under sudo (eg, sudo -H lxc ...)
• after snap installation, perform additional steps to create the system group manually, then add users to that group (eg, sudo addgroup lxd && sudo adduser foo lxd)

Device access

For many devices such as audio, UNIX group memberships are not used and instead ACLs are set on devices. Eg:

$ getfacl /dev/snd/seq
# file: dev/snd/seq
# owner: root
# group: audio
user::rw-
user:jamie:rw-
group::rw-
mask::rw-
other::---

(note ‘user:jamie:rw-’ in the above).

In the distant past, all these devices were chmod 0660 with a group specific to that device (eg, ‘audio’) or a catchall group like ‘plugdev’. While some historic groups are still found on the system (eg, ‘audio’), adding users to these groups or creating new groups and using chgrp on the devices should be discouraged on modern systems. In the not so distant past (before systemd), device access is controlled via ACLs that were handled by consolekit via ‘udev-acl’. With systemd, this is handled by ‘uaccess’ via /lib/udev/rules.d/70-uaccess.rules, 71-seat.rules and 73-seat-late.rules where the “uaccess” builtin is used to set ACLs (and some historic groups for compatibility). Since the ‘acl’ package has been seeded in Core for some time (it wasn’t at first), anything that matches Core’s uaccess rules will just work today (eg, /dev/video*).

Unfortunately there are quite a few devices that are not covered either because there isn’t a uaccess rule for it (eg, a serial-port) or because the rules don’t apply for some reason (logging in via ssh and accessing /dev/video* or /dev/snd/* (seat rules don’t apply cause not local)).

Opt-in per-snap users/groups

A lot of existing applications are designed with the notion of privilege separation and/or permanently dropping privileges to secure their code. For example, postgresql, mysql, apache, nginx, etc. Some want to start as root to bind to a port and immediately permanently drop privileges and others want to fork processes under another user to (for example) handle untrusted input.

Today, all these daemon applications must run as root and are not allowed to drop privileges. While the security policy will keep the system safe and will keep the applications isolated, the security stance of the applications themselves is lessened because their security mechanisms can’t be used under snappy (eg, consider an application that handles untrusted input that would normally run a process under a separate user-- under snappy it is the same user so if there is a bug in processing the untrusted input, that process is able to attack the main application).

Soon (now that snap-confine is re-exec’d and seccomp arg filtering work can recommence) we’ll allow snaps to use the ‘daemon’ system user/group to drop privileges. While a shared user/group, this is no worse than the shared ‘root’ user/group and will allow applications to leverage their security mechanisms for up to one group.

Snappy should allow applications to use multiple users and groups. For example, it would be desirable for a complex snap that uses the LAMP stack to be able to let apache drop to a different user than mysql is dropping to so that successful attacks against the apache process don’t give access to mysql’s resources.

Chroot environments

Sometimes it is useful to take an existing chroot setup, put it into a snap and then have the snap chroot into the directory. There are several issues with this because processes running inside the chroot will use the chroot’s /etc/passwd, /etc/group, etc. If an application within the chroot tries to privilege drop or start as a user in the chroot’s /etc/passwd, it may not map in expected ways (eg, the security policy might allow dropping to the ‘daemon’ uid on the host, but ‘daemon’ in the chroot is a different uid so it is blocked).

Classic distro vs Core

An important consideration when designing users and groups for snappy is that:

• Classic distro systems have writable passwd/group/shadow/etc databases
• Core systems have read-only passwd/group/shadow/etc databases and writable ‘extrauser’ databases via libnss-extrausers
• Classic distro systems don’t usually have libnss-extrausers installed
• extrausers cannot be used to add users to system groups (ie, it may only be used to add users in the extrausers db to groups in the extrausers db)

[jdstrand]

Discarded ideas

Dynamically created uid/gids on the device

‘Global namespace’ in Snappy and users and groups (obsolete) discusses how to use uid/gids in the global namespace by dynamically creating them during snap install, with store processes and snapd governing their use. While we do want to use the global namespace for uid/gids, dynamically creating them complicates use of the uid/gids in the squashfs since the uid/gid isn’t predictable.

User namespaces

‘User namespaces’ in Snappy and users and groups (obsolete) discusses the inherent problems and limitations with utilizing user namespaces for this feature so it doesn’t have to be repeated here.

[jdstrand]

Design

This design has been discussed with Gustavo, the snapd team and the security team. Details remain and will be discussed in this topic.

Overall design

snapd will work in conjunction with the snap store to use a ‘global ID database’. This database will provide a predictable 1 to 1 mapping of numeric ID to name. When a snap is installed using one of these IDs, snapd will consult the global database and create a group with a name that matches the global name (prefixed with snap_) and gid that matches the global ID. It then creates a user with a name that matches the global name (prefixed with snap_) and uid that matches the global ID with the primary group set to the global name. In this manner, snaps may specify these usernames/groups in their yaml and use the uid/gid in the squashfs such that the uid/gids will resolve properly via getpwnam(), getpwuid(), getgrnam(), getgrgid(), etc both inside and outside the snap.

To avoid clashes with existing uids and gids, snapd will use very high numbered IDs in the 32-bit range. Historically, early on 15-bit uids were supported and later the more common 16-bit uids. Linux and modern Unix systems now all support 32-bit uids with all but the most specialized filesystems (eg, cramfs) also supporting 32-bit uids by default (eg, ext2 and ext4 supports the non-default nouid32 option for interoperability with older (ancient) kernels). Notably, squashfs is confirmed to support 32-bit uids and gids.

To use, a snap optionally declares the username/groups it wants to use in the snap.yaml (exact yaml TBD). Eg:

name: foo
apps:
  ...
global-ids:
- foo
- bar

The store will issue a snap declaration assertion allowing the use of the name.

Upon installation, snapd will verify the snap declaration to see if the snap should be able to use the ID. If not, ignore. Otherwise snapd will first create the group (--extrausers is used on Core only):

$ sudo groupadd [--extrausers] \
    --system \
    --gid <ID from global db> \
    snap_<name from global db>

then create the user with that group:

$ sudo useradd [--extrausers] \
    --system \
    --comment "snapd global user <name from global db>" \
    --home-dir /nonexistent -M \
    --shell /bin/false \
    --uid <ID from global db> \
    --gid <ID from global db> \
    snap_<name from global db>

then snapd will generate security policy that allows use of this username and group.

At this point there are predictable uids/gids on the system that can safely be used by the squashfs, however if the snap does getent passwd foo it won’t be found because snap_foo was created. snapd should create a mapping such that inside the snap, the user doesn’t need snap_ to be prefixed, but outside it does. While the implementation details are not finalized yet, in the core/base snap, ship a glibc NSS module and add it to /etc/nsswitch.conf (will need to bind mount this on classic). This module will appropriately map the getpwnam(), getpwuid(), etc calls correctly for the snap’s processes.

Locally unasserted snap installs won’t have a snap declaration assertion telling snapd what IDs to create/use. For these, snapd will use a private ID range and will assign the username/group from this pool, locally. This range will not intersect with the global ID range. For example, if the snap.yaml had (and the snap was installed with --dangerous):

global-ids:
- foo

then snapd will take a free ID from the private range and call groupadd/useradd with this this ID.

Use case 1: System user and groups

This case is mostly handled by the current design. The snap declares what it wants to use and it can then chown, setuid, etc to the requested user/group. It does not have to prefix snap_ due to the NSS module.

What is not covered are cases like docker, lxd, libvirt, etc where the application is checking the group membership of a connecting process’ uid and upstream documentation says to add the user to a particular group (that isn’t prefixed with snap_). For example, lxd might have upstream documentation that says to run sudo adduser <user> lxd for <user> to be able to access the lxd socket. The lxd snap might have:

name: lxd
global-ids:
- lxd

which will create the snap_lxd user. Administrators then must use sudo adduser <user> snap_lxd to grant access to the lxd socket to the <user>. The question remains in whether we force upstreams to modify their documentation for snaps or to support non-snap_-prefixed users and groups. One idea would be to use system-global-ids that operates exactly like global-ids except that snap_ isn’t prefixed.

Use case 2: Device access

For things not covered by udev and systemd automatically via shipped configuration, we use setfacl to tag devices via udev rules. Eg, today serial devices are root:dialout 0660 and have no udev ACLs on them. Moving forward, we want a new backend that would be used by interfaces like the serial-port interface to specify that all serial-ports can be accessed via a particular group (eg, snap_serial-port) and the backend would add a udev rule to setfacl the serial-port devices (or use the uaccess mechanism) to ‘snap_serial-port’ such that sudo adduser <user> snap_serial-port would grant access to the serial ports for that user.

For managing the groups, there are two choices:

• put the group in the global ID database and have the backend create a group with the corresponding name and gid. Eg, have snap_serial-port in the global ID database
• have the backend dynamically create the group specified by the interface. Eg, the serial-port interface tells the backend to create the snap_serial-port group

Because the group isn’t needed by the snap itself, maintaining the group in the global ID database is not required and keeping them separate may provide a cleaner separation of the use cases.

Use case 3: Opt-in per-snap users/groups

This case is completely handled by the current design when shared global IDs (ie, where developers don’t mind that another snap might use the same IDs it requested) are ok. The snap declares what it wants to use and it can then chown, setuid, etc to the requested user/group. It does not have to prefix snap_ due to the NSS module.

The security team identified that a future improvement of the feature would be to support ‘private global IDs’ for security-concious developers who want guarantees that the IDs it uses are only used by the snap. For example, the snap.yaml might have:

name: example
private-ids:
- foo
- bar

which would create the snap_example_foo and snap_example_bar users which could only be used by this snap. To support squashfs files using predictable uids/gids, then this probably needs to be maintained in a private global ID list (to not expose private or brand snaps).

Use case 4: Chroot environments

This case is completely handled by the current design. The snap declares what it wants to use and the squashfs can reliably have files with uids/gids matching the requested global IDs.

Open questions

1. What is the starting uid/gid number of the range. Perhaps 2147483648 (ie, 2^31)?
2. What is the private uid/gid range? We don’t need many of these
3. How will snapd handle clashes with existing uids/gids on the system and the global database? useradd uses --non-unique? Panic?
4. Finalize yaml declaration
5. Make decision on whether or not to support users/groups not prefixed with snap_
6. Make a decision on using the global ID database or dynamically creating users for use case 2 (setfacl/uaccess)
7. While not discussed above, consider exposing usernames/groups from the host in the snap’s runtime (eg, the NSS module prefixes classic_ to uids on the host). Consider how to handle exact overlaps (should classic_ be prepended or not?), uids/gids that have different usernames/group names between the host and snap and usernames/group names that have different uids/gids between the host and snap
8. Consider ‘private global IDs’ and how they might be implemented

Components to modify

• snapd
• user backend for creating users/groups
• backend for setfacl/uaccess
• snap declaration verification
• support private ID ranges
• seccomp updates to allow using the uids/gids
• store
• easily view and update the global ID database (preseeded with archive grep of users and groups created in debs via adduser/addgroup/useradd/groupadd)
• easily update snap declarations
• pass global ID database to the review tools
• review tools
• warn if using uid/gid without snap decl (ie, standard human review, like with classic confinement)
• don’t complain if not -all-root anymore, but do complain if uids, gids are not in the declared list
• complain if use IDs from the private ID range

[jdstrand]

@niemeyer, @mvo, @zyga, @tyhicks - please review this topic to see if I accurately captured the sprint agreements. There are also a number of open questions we should discuss.

[jdstrand]

What is the starting uid/gid number of the range. Perhaps 2147483648 (ie, 2^31)?

FYI when deciding on an ID range: https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/1707645. It seems that while they are supported, there may be some corner cases (like lastlog) where software is written to efficiently account for them.

[niemeyer]

@jdstrand Thanks for putting this together! It does capture what we discussed well.

I’ll follow up with proposals for some of those questions soon.

[Conan_Kudo]

• Classic distro systems don’t usually have libnss-extrausers installed

I don’t even have this module in Fedora in the repositories, much less enabled as a source in nsswitch.conf. A quick lookup indicates this was a module built several years ago by someone in Debian and it lives pretty much exclusively in Debian/Ubuntu archive. The only external source we have supported out of the box is the sss NSS module (which, because it’s SSSD, does actually support multiple backends and such, but there’s nothing we have that can interface with it right now).

[jdstrand]

That is useful information. Thankfully, the extrausers module is only required in the core snap for Ubuntu Core (ie, not cross-distro). As such, it can live in the core snap and base snaps (eg, ‘Fedora base’) don’t have to worry about it. Only if someone wanted to create an alternate core snap based on a different distribution would extrausers (or an alternative) need to be considered. It would even be possible for that alternate core to skip the need entirely and make the passwd and group database read/write.

[ogra]

make the passwd and group database read/write.

Only when making all of core read write else UID/GID filesystem mappings will eventually fall apart …

[jdstrand]

My point was that would ultimately be a decision for whoever was putting together ‘Alt Core’ devices, not that it would be a recommended thing to do.

[modolo]

Hi,

I have a very strange behavior with my machine that is in AD (SSSD). For example, you can not list the files using hello-world.sh.

modolo@gsat019046:~$ snap run hello-world.sh
Launching a shell inside the default app confinement. Navigate to your
app-specific directories with:

  $ cd $SNAP
  $ cd $SNAP_DATA
  $ cd $SNAP_USER_DATA

bash-4.3$ ls
ls: cannot open directory '.': Permission denied
bash-4.3$ id
uid=265601806 gid=265600513 groups=265600513,4(adm),7(lp),24(cdrom),27(sudo),30(dip),44(video),46(plugdev),113(lpadmin),128(sambashare),131(lxd),137(kvm),138(libvirtd),997(rkt),265602411,265602421,265604408,265604415,265609914,265613528,265614005,265619450,265619467,265619643,265619720,265619721,265626944,265626955,265629586,265630854,265632970,265632975,265632993,265633732,265633737,265636639,265636643,265636657,265636659,265636673,265637680,265638362,265639476,265639496,265641502,265641854,265641855,265643223,265643255,265643431,265645967,265646099,265646372,265646461,265646560,265646775,265646882,265647071,265651567,265651622,265653636,265653673,265653733,265653757,265653855,265654408,265654550,265654552,265654561,265654890,265656369,265656370,265656371,265656375,265656378,265656379,265656380,265656382,265656384,265656386,265656388,265656390,265657145,265658322,265659423,265659533,265659589,265660036,265660834,265661748,265662107,265662110,265662117,265662118,265662131,265662136,265662139,265662143,265662145,265662543,265662739,265662740,265664437,265664505,265664999,265665490,265665669,265665758,265665833,265665886,265665887,265666570,265666594,265666614,265666618,265666619,265666620,265666621
bash-4.3$ 

I have verified that it is not possible to consult domain users!

bash-4.3$ getent passwd modolo
bash-4.3$ getent passwd root  
root:x:0:0:root:/root:/bin/bash

[jdstrand]

Today, domain users need the ‘network’ interface and the hello-world snap does not ‘plugs: [ network ]’.

To make this work in the default template (arguably something that should be done), fine-grained network mediation in AppArmor (future work) might be able to allow name lookups. Alternatively, an nss module could be made available to snaps (eg, a bind mount) that could talk to an out of process proxy on the system to do the lookups on behalf of the application.

[jdstrand]

We could also do something similar to what I proposed for NFS /home: make a core config that unconditionally allows the network interface/apparmor nameservice abstraction. I wonder how useful this (and NFS /home) would be per-snap rather than global config…

[modolo]

The lack of possibility to list users of the domain prevents me for example from completing the installation of snap libreoffice!

On the same machine if I authenticate myself as a local user, I can fire the libreoffice (snap run libreoffice) and complete the installation!

I believe the libroffice needs to check local folders (~ / .config) that are not available since the permissions become wrong.

[jdstrand]

@niemeyer - I noticed the topic title changed to ‘multiple users and groups in snaps’. This title doesn’t cover all four use cases and is somewhat confusing. The use case are (taken from the introduction):

"

1. adding system users/groups https://launchpad.net/bugs/1606510
2. supporting device access via ACLs for granting access of devices to (non-root) users. https://launchpad.net/bugs/1646144
3. per-snap opt-in users to support things such as privilege dropping/separation within snaps. https://launchpad.net/bugs/1619888
4. Supporting uids/gids in chroot environment
   "

[zyga]

The bug links above are broken, the right links are:

• https://bugs.launchpad.net/snappy/+bug/1606510
• https://bugs.launchpad.net/snappy/+bug/1646144
• https://bugs.launchpad.net/snappy/+bug/1619888