Thanks for the update, @chipaca .
Just wanted to second the point made by @morphis earlier in the thread about
WatchdogSec= being important. We’ve been asked about future (ie. 18.04) support for application based watchdog timers by one large OEM in particular.
@chipaca will your initial work include
Before= conditions? Also any thoughts as to a potential version this of snapd this will land in?
+1 to service ordering as well. I have a use case that where a snap has an mqtt broker service and a number of services that connect to the mqtt broker. Would be great if I could start the dependent services only after the mqtt broker service is started.
Work on this feature is already under way. It will be part of the next release or the one after it in the worst case.
Great, I also +1 the request for watchdog support. Currently if a start rate limit has been reached for a service, the service is put in a permanently failed state and no remedial action is possible. I’d love the option to be able to reboot the system under this scenario. Such a feature is particularly important for unattended consumer IoT applications.
The current workaround of editing the unit files through a configure hook during installation obviously breaks confinement as @hcochran has noted.
Took a quick stab at watchdog support https://github.com/snapcore/snapd/pull/4504 It seems that due to security concerns this will require some additional work though.
Is there any branch with these systemd improvements (post command, start timeout…)?
Came here to ask the same; is there a rough timelines set for implementing these? Took a look at The snapd roadmap but this doesn’t seem to be mentioned on there.
We’re still working on these improvements, and pieces have been frequently landing. We got before and after in, and @mborzecki is now working on service timers which I’m really looking forward to having as well.
It would be nice to tackle starts-with and friends next, but if the lack of something else is blocking you, please let us know the details and we might change the priorities.
FYI, service watchdog support has landed in master. The service can specify desired watchdog timeout by adding
watchdog-timeout property in its declaration:
name: foo version: 1.0 apps: i-want-watchdog: command: bin/app daemon: simple watchdog-timeout: 1s restart-condition: never plugs: [daemon-notify]
As the watchdog is actually driven/tracked by systemd, the service needs access to systemd’s notification socket. This access is provided by
daemon-notify interface, which needs to be listed in the
plugs section. Since there were some reliability related incidents regarding the notification socket in the past, the interface is not auto connected and needs connecting manually.
The snapd roadmap
Is that setting supposed to be used in snapcraft.yaml?
I added watchdog-timeout setting to an app and snapcraft started to fail with the following error:
Issues while validating None: The 'apps/ping' property does not match the required schema: Additional properties are not allowed ('watchdog-timeout' was unexpected)
snapcraft version is 2.42.1.
You may need to use
passthrough until snapcraft learns about the new setting. Eg:
apps: ping: command: foo daemon: simple plugs: [daemon-notify] passthrough: watchdog-timeout: 10s
I will try, thank you!
Any idea when the
after keywords for ordering services (from Service ordering) will land support in snapcraft so we don’t have to use
I never saw a link to a PR for before/after. Any idea what snapd release contains it? Multi-project bugs are the way to go for this type of coordination.
Is there any update on the restart-delay feature @mborzecki ? Currently we have an ugly hack where we create a shim that sleeps before starting the service proper.
Milestoned for 2.36:
While we’re waiting for start-timeout, do you have an advice on a workaround?
I’m troubleshooting an issue in microstack where the install fails due to openvswitch taking too long to start. It looks like others have run into a similar issue and solved it by setting the Timeout property in the relevant systemd .service file.
What’s the best way to do something like that, currently, in a snap?