Command line interface to manipulate services

[niemeyer]

@stub

Requiring enable/disable to be done from the inside requires every snap to implement those hooks. And implement them in a standard way so it is discoverable. I assert that most snaps will not, and will arrogantly assume that their daemons should be running,

I find it sane rather than arrogant. Someone is bundling a piece of software and saying that the given daemon is supposed to be running for the snap to work. Disabling services arbitrarily will very easily break the operation of the software, if not right away, at some point in the future that will not be easily correlated to the lack of that daemon.

@stub

However, some users need to manage their precious resources and need control beyond what the snap author provides, and at the moment the only control they have is to uninstall the snap (and destroy any related data).

That’s not true. You can run “snap disable” today and keep the data and the snap while disabling the snap.

@stub

Hmm, now I read that there is another use case for the future, where snapd starts daemons when hardware devices are plugged in or powered on.

We’ll definitely allow the snap itself to control its own daemons in the near future via snapctl.

@stub

I think the verbs should match systemd, upstart & init.d meanings or users will get confused. If you want start and stop to be persistent, rename them to enable and disable. Or have four commands like systemd:

The requirement is for the snap command line to be clear on intent, rather than it matching systemctl or anything else.

Saying “snap disable mysnap” disables the snap today, and that sounds pretty nice and expected.

With all that said, if you find it important for your use cases it seems reasonable to support that, even because administrators are already able to do that via systemctl directly anyway. So it’s mostly a convenience. I suggest the following syntax for that:

$ snap stop --disable <snap>[.<app>]
$ snap start --enable <snap>[.<app>]

[stub]

@niemeyer

However, some users need to manage their precious resources and need control beyond what the snap author provides, and at the moment the only control they have is to uninstall the snap (and destroy any related data).

That’s not true. You can run “snap disable” today and keep the data and the snap while disabling the snap.

Oh, right. Yes, that meets the use case I described.

@niemeyer

I think the verbs should match systemd, upstart & init.d meanings or users will get confused. If you want start and stop to be persistent, rename them to enable and disable. Or have four commands like systemd:

The requirement is for the snap command line to be clear on intent, rather than it matching systemctl or anything else.

The snap command line will not be clear on intent if it assigns different meanings to existing well known words. Which is fine for this original proposal, which has stop stopping the daemons rather than disabling them and the meanings match.

@niemeyer

Saying “snap disable mysnap” disables the snap today, and that sounds pretty nice and expected.

With all that said, if you find it important for your use cases it seems reasonable to support that, even because administrators are already able to do that via systemctl directly anyway. So it’s mostly a convenience. I suggest the following syntax for that:

$ snap stop --disable <snap>[.<app>] $ snap start --enable <snap>[.<app>]

This would help with another use case, which is a snap that normally runs a daemon, but also allows the user to run it manually in the foreground (eg. with debug logging switched on, or other non-standard options, or simply to stream the output through a filter). I don’t know if this is common enough to worry about for now (?).

[chipaca]

   $ snap stop --disable <snap>[.<app>]
   $ snap start --enable <snap>[.<app>]

in implementing this right now I find myself wondering whether it wouldn’t be better to (also?) support systemctl’s enable and disable verbs via enable-service and disable-service. Thoughts on this?

[niemeyer]

I’d prefer to keep the suggested forms for now. It’s extremely rare to want to disable a service without stopping it (I’ve never done that I think), so the suggested commands sound nice, solve the problem, and avoid having more commands.

It also doesn’t prevent the commands from being added later, if we find a good reason to do that.

[mhathaway]

Just jumping in on this discussion and haven’t read all the comments, but here’s my input…

I have an Industrial IoT agent that I’ve snapped that runs as a daemon with a null configuration file upon initial install. We created a snap config app that registers the agent with our system so that we can remotely administer it.

Once registered, the agent then needs to be restarted to reload an updated configuration file. I attempted to force a systemctl restart command as an operation performed by the config app. Snap won’t let this happen even when running as sudo. I can run the program standalone with sudo privileges and it works just fine. So clearly snap prevents a service restart command from being issued by a snap application. Thinking through this, I can see why one might not want to allow this, but if the user has sudo permissions on the target system, does it pose any risks i haven’t thought through?

I support at the very least simple command structure that enables the user to start, stop AND restart a snap service. It would also be useful to enable service start/stop/restart to be called by a snap app.

Cheers,

Michael

[niemeyer]

Hi Michael,

That is coming very soon and is a follow up to the work discussed in this topic. Changes landed just last week allow snapctl to be run outside of hooks, and the next piece of the puzzle is to allow these same actions that are being offer in the snap command to be run via snapctl, from within the snap itself. This should be pretty easy now that these first few steps are getting out of the way.

[mhathaway]

Thanks for working on this. In the meantime, we can ask our customers to run sudo systemctl restart on the snap manually.

Michael

[niemeyer]

We’re almost done with the feature, thanks to a lot of good work by @Chipaca. Today we had what I think is the last exchange on details of the API and user interface, which I’m copying below for awareness and for our own future benefit:

<Chipaca> niemeyer: let me know when you have 15’ to go over stuff? <niemeyer> Chipaca: Back just now <Chipaca> niemeyer: so, working from the commandline in, “snap status” as it stands in the PR is OK, yes? <niemeyer> Chipaca: Double checking <Chipaca> niemeyer: thank you <niemeyer> Chipaca: The output sounds sane… I’m just wondering a bit about the command name itself <niemeyer> Chipaca: I wonder if it feels too much like “snap status” would result in what we have as “snap list” <niemeyer> Chipaca: (the status of snaps in the system) <niemeyer> Chipaca: WDYT? <Chipaca> niemeyer: a little bit, yes <Chipaca> niemeyer: OTOH we decided against “snap service status” AFAIK <Chipaca> niemeyer: and if “snap status” feels like this, I reckon so will “snap restart” <niemeyer> Chipaca: Indeed, and we have the same sort of ambiguity on enable/disable <niemeyer> Chipaca: I guess status is fine from that angle, and the analogy of systemctl is definitely a plus <niemeyer> Chipaca: Were you thinking about something specific when you asked? <Chipaca> niemeyer: no, just walking over the changes i need to make to be sure <Chipaca> layer by layer i mean <niemeyer> Chipaca: I think we’re good on that one… I do wonder how we’ll represent timers when they come <niemeyer> Chipaca: Sounds like it’d make some sense to have them there… <Chipaca> niemeyer: I don’t understand your concern there, but that might be because I don’t know what timers are for snapd <Chipaca> I know what they are in systemd <niemeyer> Chipaca: I think that’s exactly my concern <niemeyer> Chipaca: (the fact we don’t have a clear view, which might lead to awkward corners soon) <pedronis> well people asked to have systemd(-like) timers supported for snaps <Chipaca> niemeyer: timers can be active and enabled too, fwiw <niemeyer> Chipaca: Right, that sounds sane <niemeyer> and may be stopped/etc <Chipaca> niemeyer: anyway, going one step further in, we’d have an Apps method on clients, that takes a list of things, and an options struct <niemeyer> Chipaca: Yeah, singular I think <niemeyer> AppOptions? <Chipaca> niemeyer: singular…? <niemeyer> I think that’s what we do on other cases… double checking <Chipaca> niemeyer: and, here i’m not so sure: the options struct has an All bool, which if false means just services? <niemeyer> Argh… we have ChangesOptions… we should really fix that one <Chipaca> why? <Chipaca> it’s Options <niemeyer> Yeah, maybe <Chipaca> especially given that we used to have Change and Changes, ChangesOptions and ChangeOptions are clearly different beasts <niemeyer> Yeah, that’s actually the issue <niemeyer> Often those option methods are useful in the context of a single thing <niemeyer> Let me try to find an example <niemeyer> func (client *Client) Install(name string, options *SnapOptions) (changeID string, err error) { <niemeyer> vs. <niemeyer> func (client *Client) InstallMany(names []string, options *SnapOptions) (changeID string, err error) { <niemeyer> The options are snap-related, the thing, rather than method-specific <niemeyer> thus ChangeOptions, AppOptions, etc <niemeyer> Similarly, although we have ChangesOptions, we call it ChangeSelector in one of its fields <niemeyer> With that background, yeah, indeed I’d suggest going with the singular, and eventually fixing ChangesOptions to agree <Chipaca> So Apps([]string, AppOptions)? <niemeyer> Yeah <Chipaca> ok <Chipaca> niemeyer: and would the options struct has an All bool, which if false means just services? <niemeyer> Chipaca: The opposite case feels more natural: return all by default as it’s an /apps endpoint, and allow constraining to services by providing {Service: true} <Chipaca> ok <niemeyer> Chipaca: That opens the door for us to have a special kind of service which is hidden as well <Chipaca> niemeyer: and that translates to select="", “all”, and “services” (with the two first ones being synonymous) <niemeyer> Chipaca: and which we uncloak via a future All field <niemeyer> Chipaca: Similar to what we do with snaps <Chipaca> ah, so no select=all as synonymous for ="" <Chipaca> niemeyer: ok so far? <niemeyer> Chipaca: Yeah, I’d keep just “” and “service” (again singular due to precedence in /v2/snap’s refresh) <niemeyer> s/refresh/select <niemeyer> Chipaca: For that latter use case, perhaps just “” and “service” <niemeyer> Chipaca: Again singular (precedence in /v2/snaps’s select <niemeyer> Thanks irccloud <Chipaca> niemeyer: precedent, not precedence, i assume <niemeyer> It told me it couldn’t send my messages, and then did it later <Chipaca> ok <Chipaca> but note that snaps uses adjectives <Chipaca> bah, not even <Chipaca> in snap’s select it’s “refresh” or “private” <Chipaca> uhm <Chipaca> sorry, that’s in find <niemeyer> Chipaca: Sort of… I think it’s a similar use case… “the snap is a refresh… the snap is private… the app is a service” <Chipaca> in snap it’s “all” or “enabled” <niemeyer> Chipaca: Sorry, I was really thinking of find <Chipaca> ok, singular <Chipaca> niemeyer: then, inside daemon, there’s the call to the helper that right now has a struct. I think I’ll change that to mirror the client’s options, for less surprises. <niemeyer> Chipaca: Looking <niemeyer> Chipaca: You mean the wantedAppInfo? <Chipaca> yeap <niemeyer> Chipaca: +1 <niemeyer> Chipaca: I’m tempted to suggest “snap services”… I’ve been dueling with myself for the past 30 minutes on it <niemeyer> Chipaca: Part of the question is: are timers services as well? <niemeyer> I suspect that as far as systemd is concerned, they are not <Chipaca> niemeyer: they are distinct <niemeyer> Chipaca: So will we have a {Timer: true} flag? <Chipaca> niemeyer: although in systemd a timer is associated with a service of the same name <niemeyer> /o
<Chipaca> niemeyer: that is, the timer is just a timer <Chipaca> niemeyer: when it fires, it runs the service with the same name <pedronis> I doubt we would model it that way though <Chipaca> correct <Chipaca> niemeyer: (you can change which unit it fires, but the default is the one with the same name) <niemeyer> I guess the app would be a timer and a service then? <Chipaca> niemeyer: an app would be … Daemon: timer ? <Chipaca> probably not because the service will have its own daemon: <Chipaca> niemeyer: a daemon can have a timer? <niemeyer> Chipaca: I was thinking of just having something like Schedule: <niemeyer> Chipaca: Or simliar <Chipaca> yup <Chipaca> so a service would have a timer / schedule / thing <Chipaca> makes sense to me <niemeyer> Chipaca: We might imply “Daemon: timer” in that case perhaps? <Chipaca> niemeyer: no because the service can be one-shot or notify or … <niemeyer> Chipaca: Or would it make sense for something to be a daemon and a timer? <niemeyer> Chipaca: Ah, okay, combined even in that sense… nice <Chipaca> niemeyer: man systemd.timer fwiw <Chipaca> niemeyer: also <niemeyer> Chipaca: Yeah, I’m friends with that one… have been trying to find a good syntax for ourselves <Chipaca> niemeyer: systemctl list-timers <niemeyer> Chipaca: Thanks, hadn’t seen that one <Chipaca> niemeyer: and i’d expect we’d want something similar, maybe under ‘snap timers’ <Chipaca> but, dunno <niemeyer> Chipaca, pedronis: Okay, so, what’s the experience we want? Do we show timers on “snap status/services” output? <Chipaca> because, dunno what timers are for snap <niemeyer> Chipaca: +1 on “snap timers” <Chipaca> niemeyer: if a service has a schedule, maybe include a “last run” or “next run” column? <Chipaca> niemeyer: or a “see snap timers”? <Chipaca> i mean <Chipaca> no, i wouldn’t show timers as lines in ‘snap status’ output <Chipaca> I’d show the services the timers fire, though <niemeyer> Chipaca: Or perhaps just “Schedule”, with the actual string, and leave the specialized output for “snap timers” <Chipaca> niemeyer: that sounds reasonable <niemeyer> Chipaca: Okay, thanks, that gives much better understanding of what’s to come… so back to your original question: <niemeyer> Chipaca: I think the output is mostly fine… a couple of points to ponder about: <niemeyer> 1. Should we always have the first field? That makes it more tooling-friendly (think awk, grep, …) <niemeyer> 2. Would it be useful to have Daemon with the value of that field? <niemeyer> 3. I’m slightly conflicted about the name of the “Service” header… not sure if “App” would make it more clear or more confusing <Chipaca> 1. i think yes, although i’m not sure it makes sense for it to be split <Chipaca> 2. i don’t think Daemon tells the user of the app anything they need to know, no <Chipaca> 3. i think Service is a little clearer than App, but not by a wide margin <niemeyer> Okay… for 1, should we just have it always then? <niemeyer> For 2, sounds good <Chipaca> 1., yes i think so <niemeyer> 3. Okay <niemeyer> Chipaca: Sounds like we have a plan then! <Chipaca> niemeyer: one last question: about the split of AppInfo and ServiceInfo in the client libs. It’s mainly driven by the desire for the json to be nice and clean for non-service apps, and nice and explicit for service apps <niemeyer> Chipaca: I understand, but I also see value in the conceptual flattening… we’ve already decided to make them look alike long ago, and it earned us many bonus points in terms of having plugs/etc handling not care, being able to have commands and services, etc <niemeyer> Chipaca: I think this is just being more honest about that internal representation, and passing the advantage of that flattening on to the client <niemeyer> Chipaca: It’s analogous to systemd’s “units”, if you see what I mean <niemeyer> I’m sure somebody complained about “starting a mount point”, but hey, the flat CLI is nice <Chipaca> hmm <Chipaca> niemeyer: in systemd, status of a unit (that is, the common ancestor of services, mountpoints, timers, …) makes sense <Chipaca> niemeyer: whereas our apps are very distinct beasts <Chipaca> but, i’ll flatten it <Chipaca> no worries <niemeyer> Chipaca: Not really… we do have a bunch of common ground for apps <niemeyer> Chipaca: plugs and slots, security profiles, etc etc <niemeyer> Chipaca: Thanks! <niemeyer> Chipaca: Ah, and I suggest going with “snap services”, given all of that useful background <Chipaca> ok <Chipaca> but not right now. Right now, time to walk the dog, and think of dinner, and maybe a beer <Chipaca> o/ <niemeyer> Chipaca: Sounds great, thanks for the chat… I’ll copy that conversation into the forum <Chipaca> tks <Chipaca> that’s another thing that’s harder to do in a hangout

[niemeyer]

As a follow up from a point raised by @pedronis elsewhere, perhaps we can improve a bit the output of “snap services” so reading it is easier to tell what the output means at a glance.

The output today looks something along these lines:

$ snap services
Snap   Service  Enabled  Active
foo    bar-00   false    false
foo    bar-01   false    true
foo    bar-10   true     false
foo    bar-11   true     true

A couple of issues with this:

• on a long list, it won’t be clear what true/false means anymore
• when grepping, it will be hard to tell because headers are gone
• it’s not very visually appealing (wall of bools, as @pedronis pointed out)

So, here is a suggestion which tweaks the output slightly:

$ snap services
Snap   Service  Startup   Current
foo    bar-00   disabled  inactive
foo    bar-01   disabled  active
foo    bar-10   enabled   inactive 
foo    bar-11   enabled   active

How does that look?

[chipaca]

At the standup today we talked about how having a list of booleans wasn’t particularly friendly, and that while any of the four combinations were allowed, we could present the ones that were anomalous.

@niemeyer said he’d work on getting some options (and he beat me to it, posting it anyway), but meanwhile I thought I’d give it a try and see. First, what we’d have as the PR stands:

Snap          Service      Active  Enabled
another-snap  svc1         true    false
another-snap  svc2         true    true
some-snap     service-one  false   true
some-snap     service-two  false   false

now with the discussed “disabled and active is weird, and enabled and inactive is weird”, but the status in English:

Snap          Service      Status
another-snap  svc1         disabled but active
another-snap  svc2         active
some-snap     service-one  enabled but inactive
some-snap     service-two  disabled

or a more machine-like status,

Snap          Service      Status
another-snap  svc1         disabled,active
another-snap  svc2         active
some-snap     service-one  enabled,inactive
some-snap     service-two  disabled

thinking forward a little bit, to when something might be enabled and inactive because it’s fired by a timer,

Snap         Service      Status
timely-snap  service-two  enabled, next run at 2pm

or

Snap         Service      Status
timely-snap  service-two  enabled, runs daily@2pm

i think either works.

[niemeyer]

For observers, we were both in the same conversation elsewhere, and ended up writing down slightly different proposals out of it.

@chipaca

or a more machine-like status,

Snap Service Status another-snap svc1 disabled,active another-snap svc2 active some-snap service-one enabled,inactive some-snap service-two disabled

I wrote that exact one myself before sending the proposal above. I quite like it as well, but ended up deleting and sending the one above that leverages the same logic, with enabled/disabled and active/inactive explicit terminology, because it assumes less in the sense that snaps may choose to use manually started services based on internal configuration. In that sense, we’d be highlighting a specific state as awkward but for the snap itself it’s actually normal.

Other than that, I think we’re pretty much aligned.

[chipaca]

to be clear, I prefer your proposal (although extending it for services with timers will probably result in a new column) to either of mine.

[chipaca]

One (hopefully last) quirk: as defined above, snap restart foo will start foo if it’s not running, whereas snap restart --reload foo will not. Documenting that is tedious, and using that is confusing.

This is because --reload changes systemctl restart (which starts services that are stopped) to systemctl try-reload-or-restart (which doesn’t). There exist also try-restart and reload-or-restart, either of which would make this problem go away. Could we change it?

[niemeyer]

Thanks for catching that. +1 on using reload-or-restart instead. We can make the “don’t do anything if not running” an orthogonal flag in the future, which would act both on restart and restart --reload.

I would not name that “–try”, though. That terminology seems pretty confusing.

[chipaca]

this is on master. Might need a second pass to improve the task descriptions.

[mborzecki]

Necrobumping an old topic.

Both sockets and timers are in snapd now, but are not visible in snap services output, nor can be manipulated using snap(-ctl) <start|stop|restart> command.

I tried to experiment with this a bit and ended up with something like this:

$ snap services
Service                                       Startup   Current
lxd.activate                                  enabled   inactive
lxd.daemon                                    enabled   inactive
lxd.daemon.unix.socket                        enabled   active
test-snapd-timer-service.random-timer         disabled  inactive
test-snapd-timer-service.random-timer.timer   enabled   active
test-snapd-timer-service.regular-timer        disabled  inactive
test-snapd-timer-service.regular-timer.timer  enabled   active

Sockets end up with <socket-name>.socket suffix, while timers (there is only one per service), are suffixed with .timer.

The the manipulation could be done like this:

# stop the socket
$ snap stop lxd.daemon.unix.socket

# start the service
$ snap stop lxd.daemon.unix

# start the timer
$ snap start test-snapd-timer-service.regular-timer.timer

# logs of the timer (i.e. when it fired)
$ snap logs test-snapd-timer-service.regular-timer.timer

No need for extra --timer|--socket switches. One can just copy & paste the service/socket/timer name.

I already have some changes and can propose a branch. I’m open to suggestions.

[ijohnson]

In my branch here, I implemented the ability for snap stop and snapctl stop to stop/disable the socket and timer services alongside the original service when stopped so that snap stop lxd.daemon stops both the associated service and the socket (and same thing for timers).

What is an example use case of being able to start/stop the sockets/timers independently of the service?

[niemeyer]

@mborzecki That proposal makes sense if you look from the perspective of systemd units, but this feels like leaking an implementation detail that is not necessarily useful at a higher level.

If we ignore the implementation altogether, we have applications with a timer. That’s pretty straightforward… the application will be run in the schedule, as defined. From that angle, we can also extrapolate the meaning of commands on the application:

• stop - Seems easy… stop the application if it’s running.
• stop --disable - Also feels straightforward… stop it now, and prevent it from running again.
• start - Not so obvious… I’d make this run the service once, so that the next one makes sense too.
• start --enable - Run it once and enable the timer so it runs in the future as well.

@ijohnson

What is an example use case of being able to start/stop the sockets/timers independently of the service?

Right, @ijohnson’s question is spot on.

[chipaca]

I agree with @niemeyer about not exposing timers / sockets as toplevel addressable things via services. I would, however, tweak snap services output to better support services that are socket-activated, or have timers. E.g. for timer-based, have a note about when it’ll next be run (or if the timer has been disabled); for socket-activated, mention whether the socket unit itself is enabled/disabled. This could be done via a Notes column.

[niemeyer]

@Chipaca Agree with most of it… just not sure about the enable/disable of socket. It seems to fall into a similar realm of the service/timer separation. How much is the socket actually independent from the service, and for what purpose? Might be good to at least postpone that part until we understand that better.