I am an Ubuntu Desktop (16.04.3) user and I have recently installed some snaps. I am doing some research to understand their effect on my system and I found out about this.
I was really surprised by this and I have to say I am a bit disappointed. The obvious similarities to Windows 10 and it’s problems have already been pointed out. More surprising to me is the project management approach of controlling the user (through restricting the user’s options). This is against what free software is about.
It’s not that I don’t want to keep my laptop updated. Whenever I start it it’s one of the first things that I do. But having that control taken away is very off putting.
I can think of cases where I would want to delay updates. Some have been mentioned already. It’s true that they are the minority of my usage. But it is also true that they are there. So I would like to put the issue on a statistics point of view.
Let’s say that snaps are successful and get deployed at 500 million devices until 2020. (According to some lazy googling IoT devices in 2020 were expected to be upwards of 20bil so the market share I picked ain’t huge. Didn’t check the validity of those estimates though). Let’s also say that you have designed snap forceful updates so well that you can successfully cover 99.9% of use cases. This will leave you with 500.000 devices (!!!) that have use cases you are not addressing and will be forced to hacks like blocking connections to the update servers etc, which are even worse!
If a knowledgeable user with root access wants to disable updates he can do it, you might as well give him an option to do it in a way that doesn’t further compromise his system.
The solution, as far as I can see, is to add attrition to the option you don’t want to be enabled. You know the power of defaults. If the update option is on most people will leave it at that. On your enterprise customers you can explicitly say in the support contract that this option will void any support contract guarantees. You can output a big warning whenever a user connects to the device urging them to turn automatic updates on.
If after all your measures somebody insists on disabling updates you should allow him. He will be using an unsupported configuration, he may very well be doing it wrong, but in the end it is the user’s prerogative (in the context of open source where you are supposed to own your computing devices instead of renting them through an EULA).
Please reconsider this.
TLDR:
Don’t try to control the user.
If you gain serious market share there will be a lot of users with valid use cases that need this option.
thanks for outlining this again, but I might be a “special” user. I don’t really care for myself as I’m on the developer side, but I care for the users of my snap who complain about a “crash” (not a crash, it’s actually the restart) every time I push a new snap of my application.
The best I can do is suggest your solution to my users, but I would rather have a real solution.
I’ve outlined the above case as frequently the push back was to do better testing and slowly promoting the snap from edge to stable. In this case this would not have helped.
A workaround like you suggest would work. Downloading the snap and installing it manually is another (ugly) workaround to avoid updates for a specific snap.
Updates happen on scheduled windows, not every time you push your snap. We’re working right now (@mborzecki is looking into this already) into pushing the scheduling to a monthly basis, which means people can schedule exactly what they want inside the month.
The primary goal is not to take control away from the user altogether, but to strongly encourage the practice of regular maintenance. Per conversation above, we’re working on multiple features that better support this idea, wider scheduling being one of them, and epochs being another, both in progress (health checks too, but not yet in progress). We also haven’t ruled out doing exactly what you suggest, and simply introducing a global refresh shutdown flag. The reason we haven’t done that yet, again, is because once we go there we cannot take it out, and that one chance of trying to establish new practices around this problem is gone.
If you can do what @denis suggests for the time being for the unbearable cases, and stick with us while trying to see if we’re able to move forward with the paradigm of automatic refreshes being the common rule, then we’d also be very happy to continue learning from your experiences (positive and negative) with those mechanisms so we can jointly understand how far we can take it.
^^ This behavior would be desirable for our application. We have an auto-update built in to our application that allows us to update the app without updating the snap. We allow our users to set the maintenance window that we would like any updates to conform to. Even though we don’t have to update the snap to update our application, there may be times when an update to the snap is necessary and we’d love to be able to have the snap refresh conform to the maintenance window set for our app.
We don’t have a timeline on that part of the feature just yet. The high-priority changes which are already in progress aim at increasing control on the user side.
Maybe I’m late for this discussion, but I want the developers to know how necessary is to give the users the option to disable the autopdates. Imagine for a moment that you live in a 3rd world country where 40 people share a 1Mbps link and suddenly all of them start downloading the ubuntu-core update. That’s the case where I work. Microsoft Windows 10 is banned in our environment until we hack it and prevent it from autoupdating. The process is complicated, but doable. With snapd is hardcoded, so our only option would be to download the sources, disable the autoupdate in the code, compile, deply, etc and that’s insane. Implementing scheduling will take you time, giving the users a way to disable autoupdates will take you a couple of hours at most and will make everybody happy. Give the users freedom. Thank you for the all hard work and please think again about this issue.
‘everybody happy’ except when some vulnerability which would’ve got fixed by updates gets exploited. It seems monthly refresh scheduling will be out by the end of January or so (there’s no release dates for snapd 2.31 yet but it looks like it’ll be in that) or a few weeks earlier if you refresh to the snapd 2.31 beta when it comes out. See The snapd roadmap
I agree with your sentiment though, it would be faster to implement an automatic refresh disable switch, and would respect users’ control…but it seems the snappy developers would rather go for the long haul here and instead try and meet all users’ needs without implementing that switch. Do they need to allow refreshes to be capped at a certain speed? Managed across all your devices at a higher level? Give a longer refresh window than one month? I’m sure they will seriously consider and implement suggestions that don’t involve an off switch, so this is the place to make them!
By everybody I mean, everybody who wants to disable it for some reason. While I support 100% reasonable defaults, I also think people should be able to override them if necessary. Life’s entropy can prove any configuration, default or decision wrong. While updates are usually good there are a thousand edge cases where they are not desirable or can be even dangerous.
The current thread is about disabling the automatic refresh, so this should the right place to suggest a switch off. If snapd developers don’t want to listen and want to continue thinking their users are children who cannot handle the consequences of disabling automatic updates, then probably many will stop using it.
Another option is also to point api.snapcraft.io to 127.0.0.1. Sysadmins with low bandwidth on their networks could also implement it for their whole networks at the DNS servers.
Perhaps it is advisable to consider creation of a store proxy which allows interception of legitimate store requests and caching the binaries delivered in the response while allowing everything else to occur synchronously?
Probably not yet, someone needs to add something under #doc ideally.
To see now how to use it, see here for the command and then use the appropriate syntax here for the end of that command (assuming there weren’t changes in the PR that deviated from that design…)
There’s some documentation on it already actually, under the core configuration options. The monthly style isn’t there yet, though, because it’s only coming in this next release. We need to update it to cover that indeed.
Chiming in with some real-world experience - we were just victims of an automatic update that bricked a bunch of our devices in the field (we’re using the Dell IoT 3001 gateways, and an update caused an issue that disabled cellular connectivity to our units). We cannot re-establish communication with these devices without performing a hardware reset.
This happened in the middle of the night (always fun!), so our developer team was awoken around 3AM to triage, and then our operations team spent the entire day fielding customer complaints.
Overall I’m a big fan of the philosophy of encouraging regular software updates as a default. But forcing users to accept updates, even with features like scheduling, strikes me as unbearable. We should be able to control the software that runs on our systems and not be beholden to our snap overlords.
I shudder to think about this in the context of a mission-critical / life safety application.
I guess what the snappy design team are assuming is that, in theory, it’s possible for the upstream to test for most edge cases before pushing an update and it’s possible for the user to put time and effort into scheduling updates at a certain time and that they’ll be able to test for most problems before they actually happen - since this is what users do before upgrading software in any case. Even in mission-critical / life safety applications, at some point they update their software, right? How do they do so? By upgrading at a certain time and by extensively testing the upgrade. Why doesn’t the scheduling system allow this to happen?
I know there’s a deeper issue here too of:
It should be noted that snapd is open-source and can thus be forked and modified. The snap design team will hate it (so much that they may delete this post), but it’s possible we could undo the hardcoding to the snappy store and the automatic refreshes and publish the changes as a soft fork. We need someone with the time to code and maintain this though, do you know anyone? Note that said soft fork would probably be less reliable than the automatically refreshing and well-supported snapd so this would be more a proof-of-concept than anything… I realize this flies sharply into the face of much of the snappy project goals but equally, currently, snappy is flying in to the face of many user demands and deep-seated beliefs…
Can you share more details of what actually happened with us? If so, let’s have it in its own topic if you don’t mind, so we keep the conversation here in context.
Then, rest assured the picture is not so bad as it may look like.
Here are some details:
Scheduling of refreshes has been supported for a very long time. The fact it refreshed at 3AM means it was not configured to refresh at a more convenient time.
Starting with 2.31 snapd can also schedule on a specific window over the entire month, so you can also pick the exact day and time the team will be available to support it.
We’ll soon start working on health checks, which is a feature that will enable snaps to report whether an automatic update worked or not, and that will immediately revert if there are problems.
We’ve recently landed the repairs feature, which is an emergency system that allows delivering signed documents into systems to unbrick them. That’s made in a signed and very visible way so that neither us nor anybody else can sneak in.
Dell devices make use of a validation feature for the main snaps (core, kernel) that only release fundamental snaps into the system once they are approved by testing.
Dell device has further control of refreshes because they have their own management solution. I can hook you up with our commercial team if you’d like more details about it.
By all means, please talk to us if you plan to use snaps to control an airplane or a heart monitor.
So again, I’m sorry for your incident, and rest assured we’re working to make automatic updates not be painful for anyone, user, publisher, or manufacturer, but we do believe there’s important value in strongly encouraging everyone in that picture to not have to manually act to get problems fixed.
If you are interested, we are happy to work with you to improve this picture even further.
