^^ This behavior would be desirable for our application. We have an auto-update built in to our application that allows us to update the app without updating the snap. We allow our users to set the maintenance window that we would like any updates to conform to. Even though we don’t have to update the snap to update our application, there may be times when an update to the snap is necessary and we’d love to be able to have the snap refresh conform to the maintenance window set for our app.
We don’t have a timeline on that part of the feature just yet. The high-priority changes which are already in progress aim at increasing control on the user side.
Maybe I’m late for this discussion, but I want the developers to know how necessary is to give the users the option to disable the autopdates. Imagine for a moment that you live in a 3rd world country where 40 people share a 1Mbps link and suddenly all of them start downloading the ubuntu-core update. That’s the case where I work. Microsoft Windows 10 is banned in our environment until we hack it and prevent it from autoupdating. The process is complicated, but doable. With snapd is hardcoded, so our only option would be to download the sources, disable the autoupdate in the code, compile, deply, etc and that’s insane. Implementing scheduling will take you time, giving the users a way to disable autoupdates will take you a couple of hours at most and will make everybody happy. Give the users freedom. Thank you for the all hard work and please think again about this issue.
‘everybody happy’ except when some vulnerability which would’ve got fixed by updates gets exploited. It seems monthly refresh scheduling will be out by the end of January or so (there’s no release dates for snapd 2.31 yet but it looks like it’ll be in that) or a few weeks earlier if you refresh to the snapd 2.31 beta when it comes out. See The snapd roadmap
I agree with your sentiment though, it would be faster to implement an automatic refresh disable switch, and would respect users’ control…but it seems the snappy developers would rather go for the long haul here and instead try and meet all users’ needs without implementing that switch. Do they need to allow refreshes to be capped at a certain speed? Managed across all your devices at a higher level? Give a longer refresh window than one month? I’m sure they will seriously consider and implement suggestions that don’t involve an off switch, so this is the place to make them!
By everybody I mean, everybody who wants to disable it for some reason. While I support 100% reasonable defaults, I also think people should be able to override them if necessary. Life’s entropy can prove any configuration, default or decision wrong. While updates are usually good there are a thousand edge cases where they are not desirable or can be even dangerous.
The current thread is about disabling the automatic refresh, so this should the right place to suggest a switch off. If snapd developers don’t want to listen and want to continue thinking their users are children who cannot handle the consequences of disabling automatic updates, then probably many will stop using it.
Another option is also to point api.snapcraft.io to 127.0.0.1. Sysadmins with low bandwidth on their networks could also implement it for their whole networks at the DNS servers.
Perhaps it is advisable to consider creation of a store proxy which allows interception of legitimate store requests and caching the binaries delivered in the response while allowing everything else to occur synchronously?
Probably not yet, someone needs to add something under #doc ideally.
To see now how to use it, see here for the command and then use the appropriate syntax here for the end of that command (assuming there weren’t changes in the PR that deviated from that design…)
There’s some documentation on it already actually, under the core configuration options. The monthly style isn’t there yet, though, because it’s only coming in this next release. We need to update it to cover that indeed.
Chiming in with some real-world experience - we were just victims of an automatic update that bricked a bunch of our devices in the field (we’re using the Dell IoT 3001 gateways, and an update caused an issue that disabled cellular connectivity to our units). We cannot re-establish communication with these devices without performing a hardware reset.
This happened in the middle of the night (always fun!), so our developer team was awoken around 3AM to triage, and then our operations team spent the entire day fielding customer complaints.
Overall I’m a big fan of the philosophy of encouraging regular software updates as a default. But forcing users to accept updates, even with features like scheduling, strikes me as unbearable. We should be able to control the software that runs on our systems and not be beholden to our snap overlords.
I shudder to think about this in the context of a mission-critical / life safety application.
I guess what the snappy design team are assuming is that, in theory, it’s possible for the upstream to test for most edge cases before pushing an update and it’s possible for the user to put time and effort into scheduling updates at a certain time and that they’ll be able to test for most problems before they actually happen - since this is what users do before upgrading software in any case. Even in mission-critical / life safety applications, at some point they update their software, right? How do they do so? By upgrading at a certain time and by extensively testing the upgrade. Why doesn’t the scheduling system allow this to happen?
I know there’s a deeper issue here too of:
It should be noted that snapd is open-source and can thus be forked and modified. The snap design team will hate it (so much that they may delete this post), but it’s possible we could undo the hardcoding to the snappy store and the automatic refreshes and publish the changes as a soft fork. We need someone with the time to code and maintain this though, do you know anyone? Note that said soft fork would probably be less reliable than the automatically refreshing and well-supported snapd so this would be more a proof-of-concept than anything… I realize this flies sharply into the face of much of the snappy project goals but equally, currently, snappy is flying in to the face of many user demands and deep-seated beliefs…
Can you share more details of what actually happened with us? If so, let’s have it in its own topic if you don’t mind, so we keep the conversation here in context.
Then, rest assured the picture is not so bad as it may look like.
Here are some details:
Scheduling of refreshes has been supported for a very long time. The fact it refreshed at 3AM means it was not configured to refresh at a more convenient time.
Starting with 2.31 snapd can also schedule on a specific window over the entire month, so you can also pick the exact day and time the team will be available to support it.
We’ll soon start working on health checks, which is a feature that will enable snaps to report whether an automatic update worked or not, and that will immediately revert if there are problems.
We’ve recently landed the repairs feature, which is an emergency system that allows delivering signed documents into systems to unbrick them. That’s made in a signed and very visible way so that neither us nor anybody else can sneak in.
Dell devices make use of a validation feature for the main snaps (core, kernel) that only release fundamental snaps into the system once they are approved by testing.
Dell device has further control of refreshes because they have their own management solution. I can hook you up with our commercial team if you’d like more details about it.
By all means, please talk to us if you plan to use snaps to control an airplane or a heart monitor.
So again, I’m sorry for your incident, and rest assured we’re working to make automatic updates not be painful for anyone, user, publisher, or manufacturer, but we do believe there’s important value in strongly encouraging everyone in that picture to not have to manually act to get problems fixed.
If you are interested, we are happy to work with you to improve this picture even further.
I understand the problem with users holding back upgrades for very long, but there are many other usecases here. I for example is quick to deploy upgrades, but I strongly dislike forced background upgrades.
I want to learn about what’s new before deciding to upgrade. 99/100 times I decide to upgrade right away, but I really need to know what the upgrade brings to the table. Do the new software version remove compatibility with old files or hardware, then I may want to resolve it before upgrading. Is there a all new UI, then I want to learn about it before launching my app. Is the frustrating bug I’ve been avoiding fixed, then I want to learn about it immediately so I can stop avoiding it. Did a upgrade introduce a bug then I would probably have no idea about the background upgrade at all (unless something dramatically changed) as I don’t keep checking the version number every time I launch my apps, filing a regression bug against it would probably be one of my very last thoughts. Etc. Most things with snaps are fantastic and snaps are making Linux a whole lot better in many areas, but the upgrade process needs to be at least as transparent to the user as non-snap upgrades. One of the main reasons I moved from Windows to Linux was because of suprice upgrades, introducing this again will force people like me to stay away from using snaps. I hope this makes you better understand one of the reasons to why some of us are (unwillingly) avoiding snaps.
The epochs feature means that you’ll only be updated if the new snap is compatible with the old snap’s files (though this isn’t out yet), this should help with that concern (assuming snapcrafters use epochs properly).
I know this is a bit late in the process but I think our scenario might add to the discussion.
We run a SAAS system for a large number of clients. Each client has their own VM instance.
Many of our clients run a 24 x 7 operation.
Many of our clients are also hypersensitive to any system changes.
The result is that we need to have complete control over when any upgrades happen to a system (including any third party snaps that our stack depends on).
We also need to apply upgrades (such as emergancy patches) to specific systems as and when needed.
So whilst I understand the inclination to ensure that all snaps are always up to date, taking the control out of system administrators hands seems to be a mistake.
I completely agree that the default snap install should auto update.
This handles the most common case where the user either doesn’t know or doesn’t want to manage updates.
The second scenario where you can schedule updates (say micro updates) each Tuesday is also nice.
But I have to be honest I don’t see any way around a global dead switch for environments like ours that need to tightly manage upgrades.
I’m happy if the switch is hard to turn on and not obvious, but if I need the control then the control should be mine.
We can’t possibly predict every possible way that snaps are going to be used and the requirements for upgrade paths, so rather than dictating how things are we should provide great defaults but allow experienced people to manage their systems how they see fit.
