Can't mask or disable snap services: why /etc/systemd/ instead of /lib/systemd/?

We have a number of complaints from users of our snap that it’s difficult to disable the timer that our snap ships with.

Indeed:

  1. Using systemctl disable on the timer appears to be reverted upon snap refresh, and
  2. It is not possible to use systemctl mask to disable the timer, becaue the unit file is stored in /etc/systemd/system directly, instead of symlinking to /lib/systemd/system.

What is the rationale for using /etc/systemd?

It would be nice to be able to provide a simple way for our users to disable our snap’s timer/service, as is the case if they install it using a deb/rpm. At the moment, the only advice we can think of to provide is a convoluted systemd override file to unset the timer’s OnCalendar property.

1 Like

From a purely systemd and Linux maintenance perspective, I agree: /lib/systemd seems like a better place for snap service unit files. However, putting myself in the shoes of a snapd maintainer, I would argue that systemd is meant to be an implementation detail: snapd abstracts a lot of systemd concepts away so that it can possibly run on systems that lack systemd. As a result, I would recommend against treating the snapd-created service names as public API, and lean on (hopefully) snapd-provided functionality instead. As an example, instead of using systemctl disable, use snap stop --disable. This probably won’t cover every use-case, but missing use-cases are worth logging a bug and having a discussion.

From the referenced certbot bug:

sudo snap stop --disable certbot.renew

Only thing, in my attempts to verify, this doesn’t seem to actually work. The timer still triggers the service and the service still launches Certbot.

That sounds like a legit snapd bug.

1 Like

Another point on /etc/systemd vs /lib/system, is that there’s no guarantee that /lib is located on a writable filesystem. It appears to be a widely accepted pattern these days that /lib is the distro defaults (and hence could even be on an immutable, atomically upgraded filesystem), while all overrides or customizations happen in /etc.

1 Like

Thanks! The point about systemd is well-taken.

Initially when I investigated this, snap stop --disable didn’t seem to properly stop the underlying systemd timer. When I check it now, it does seem to disable the timer. That is good.

The other aspect of this is whether disabling the timer survives a snap refresh.

When I look, doing a manual refresh (switching between --stable and --beta channels) seems to re-enable the systemd timer. Is that intentional or not? I’m not sure.

If automatic refreshes re-enable the timer, that is going to be an issue for our users.

I’m waiting for our next --beta release to see what happens to the timer on my test server after an automatic refresh (done: automatic refresh re-enables the timer :cry:) , but any hints about whether the enable/disable state is supposed to survive refreshes, would be appreciated!

This could be a bug. I recall some changes that kept the services disabled across refreshes, but maybe that didn’t cover sockets and timers. Worth filing in LP.

1 Like

Yeah, it seems to make sense that services and timers that the user specifically disabled should stay disabled, regardless of what snapd does (e.g. refreshes), until the user enables them again. Please do log a bug, @_az.

Thank you, filed at LP https://bugs.launchpad.net/snapd/+bug/2002580.

1 Like