Socket activation support

#1

Can we get eyes on LP: #1612440?

As far as I recall there is code in snapd that handles socket activation, but many moons ago it was decided that the syntax needed rework so we never exposed it in snapcraft.

The aforementioned bug proposes what follows:

The current implementation in snapd is done with two options on the app:

  • socket (boolean, requires listen-stream)
  • listen-stream (the systemd listen-stream string)

This should probably be changed to something more like:

sockets:
 lxd-unix:
   listen-stream: /run/lxd/unix.socket
   socket-group: lxd
   socket-mode: 0660
 lxd-tcp:
   listen-stream: [::]:8443
Socket activation for well-known locations
#2

Errata:

Seems the snapd codebase does not have support for socket activation any more as seen in PR: #2369 which makes a comment about the bug I brought up.

#3

Note that ‘socket-group’ would require Multiple users and groups in snaps

#4

Yes indeed, that leaves the question of wanting to grow the feature in steps or all in one sweep. I would say that adding socket activation without users and groups would be a no worse situation than today with the added benefit of non essential services out of the process list.

That said, I am not aware of the grand design for this, so it might be more work to add this first depending on the amount of refactors planned to support Multiple users and groups in snaps

#5

If you’re going to support multiple activatable sockets, it would be helpful to make sure they are configured in a predictable order so that the daemon can use sd_listen_fds() rather than sd_listen_fds_with_names().

So maybe use an array rather than a mapping in the yaml?

#6

To give a bit of extra context. LXD will not be able to transition to snap-only until we have socket activation.

LXD is pre-installed on all server Ubuntu systems and it’s therefore unacceptable for it to be running on systems which don’t actively use it.

As we intend to switch to snap-only LXD early in 18.04, we really need to have this sorted out.

#7

If the configuration format is agreed upon we (LXD) could work on implementing at least the minimal set of features that we need (what was there before + socket permissions), the rest could be added later.

#8

According to https://www.freedesktop.org/software/systemd/man/sd_listen_fds.html, it seemssd_listen_fds() returns file descriptors in an arbitrary order. I’m not sure how you can ensure ordering with multiple sockets.

#9

Looking at the systemd.socket man page, it seems that sockets defined in the same unit file will be provided in the order defined, but if a service depends on multiple socket units there is no guarantee which order those units will be provided in.

Maybe it isn’t as important as I thought.

#10

you can go back from the fd to the name (with getsockname(2)) to know which is which , that’s what snapd itself does with its sockets for example

#11

I pushed a PR at https://github.com/snapcore/snapd/pull/3916 implementing the basic options (ListenStream and SocketMode) for socket activation

#12

just for the laughs btw …

using socat in a wrapper script also gets you nicely working socket activation :wink:
(this is particulary using port activation though, but socket is possible as well)

#13

Should we define each socket in its own .socket file or have a single file associated to the .service one?

The former allows finer control on each socket; for instance the LXD snap could disable the socket on 8443 if only the local socket is enabled.

If we go with a single .socket file, then there is no real use for the socket name (lxd-unix and lxd-tcp in the example), so we could make the sockets stanza just a list, but all sockets would be started/stopped together by systemd.

#14

@ack, How would this feature work together with snap set to allow administrators to change the addresses the service listen on?

#15

@ack Per comments in the review, your approach of multiple sockets sounds nice.

A few more details to figure, as described in the review.

#16

@chipaca Good question… I think it’s on us to answer that.

#17

I don’t know much about snap set, but the current implementation won’t interact.

Basically it lets you define a set of sockets that would cause the service to be started. What I mentioned about having separate socket files would possible let the service disable the .socket if it doesn’t actually listen on a specific port.
IOW you’d have to define all possible sockets that the service could listen on.

#18

@ack Yeah, don’t worry too much about that. There are a few ideas to explore in terms of how to allow people and even the snap itself to tune the default value of this setting at runtime, but we can figure this out in a follow up and so it’s not a blocker for this to land.

The only real blocker we have now is figuring out how to define the snap filename in a safe way, per the review.

#19

As a first pass can we just restrict the socket paths to be under the snap’s writable paths?
That would effectively be either:

  • /var/snap/NAME/current/
  • /var/snap/NAME/common/

Both of those should always be entirely safe and avoid any potential conflict with the host. Having this initial restriction would unblock LXD (which binds /var/snap/lxd/common/lxd/unix.socket) and would be compatible with allowing other paths (/run or such) down the line either through interfaces or another mechanism.

@jdstrand you may have some opinions on this :slight_smile:

#20

@stgraber That would be the best approach actually, as it would effectively allow systemd to create anything that the snap might create anyway. The question is just how to verify it easily from that location. The vague option we discussed was to have some sort of support on snap-exec to answer that question, and then do the typical snap run → snap-confine → snap-exec dance to get an answer. It feels a bit fiddly, but I can’t see good alternatives yet. If we go that route, systemd socket files have an ExecStartPre option that runs before the socket is created. Assuming it aborts the socket creation if that command fails, we might leverage it.