What system is this? There is an old bug that leads to this and as such this typically only shows up on systems that don’t re-exec or are too old to know to install the snapd snap in such a situation
It’s a bionic instance in Digital Ocean. Not a new instance, however, it had no snaps on it until that moment. It also only has security updates enabled, so could the problem be related to this?
Yes, could be. If you’re able to get this pristine system again, first try upgrading snapd from the normal bionic-updates pocket and then see if it happens again
$ lxc launch ubuntu:bionic -e
Creating the instance
Instance name is: evolved-mule
Starting evolved-mule
$ lxc exec evolved-mule su - ubuntu
ubuntu@evolved-mule:~$ dpkg -l | grep snap
ii snapd 2.42.1+18.04 amd64 Daemon and tooling that enable snap packages
ubuntu@evolved-mule:~$ sudo apt install snapd=2.37.4+18.04.1
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
libfreetype6
Use 'sudo apt autoremove' to remove it.
Suggested packages:
zenity | kdialog
The following packages will be DOWNGRADED:
snapd
0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 0 not upgraded.
Need to get 14.7 MB of archives.
After this operation, 2306 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
<snip>
ubuntu@evolved-mule:~$ dpkg -l | grep snap
ii snapd 2.37.4+18.04.1 amd64 Daemon and tooling that enable snap packages
ubuntu@evolved-mule:~$ snap list
No snaps are installed yet. Try 'snap install hello-world'.
ubuntu@evolved-mule:~$ sudo snap install stream-monitor
stream-monitor 0.1 from Kyle Fazzari (kyrofa) installed
ubuntu@evolved-mule:~$ snap list
Name Version Rev Tracking Publisher Notes
core18 20200124 1668 stable canonical✓ base
stream-monitor 0.1 3 stable kyrofa -
ubuntu@evolved-mule:~$ snap interfaces
Slot Plug
- stream-monitor:network
I’m afraid “just update snapd” isn’t a great option here, at least not one that scales. It’s standard practice to only have security updates enabled in production, which means there are probably tons of servers out there with this same issue (although I will caveat and say it’s probably not super common for such servers to suddenly have a snap installed… but this issue keeps popping up). Perhaps we should consider pushing an update to -security so this doesn’t continue biting folks (particularly as core18 becomes the standard base). It’s not pretty or ideal, but it’s not unprecedented either. @mvo, how would you feel about that? @alexmurray how about you?
Update:@mvo and I chatted in IRC about this. He is, properly and understandably, concerned with abusing -security like this. He mentioned that the situation would be better if snapcraft injected an "assumes: [snapd2.41] into core18-based snaps. At least then users would be greeted by an error message instead of just a broken snap. @sergiusens, does this seem doable? It doesn’t completely eliminate this issue as it requires that the snap being installed was built with a snapcraft that does exactly that, but it seems like a good step.
Indeed, it’s not a solution, but seems like a sensible step. Might need to give the error message a UX review though.
To be clear, the assumes checks the re-exec’d snapd, right? i.e. this would have no effect on existing installs with an old snapd deb but that already have the core snap installed?
I’m not fundamentally against “abusing” -security if this can solve this issue in a more usable manner - the current situation definitely seems sub-optimal and whilst I like the idea of ensuring snapcraft can add appropriate assumes but again as this will only work on newly rebuilt snaps that use snapcraft this will be a slow process, whilst just pushing an updated snapd via -security would be a much quicker solution. However, if we did go down this path I would not want to set a precedent for this sort of thing since clearly this is an abuse of -security as this is not a security issue.
Whilst there have been security issues in snapd in the past, there are no known unresolved ones as far as I am aware at the moment. -security should really only be used for updates which fix a known security issue OR a regression caused by a previous update for a known security issue. We do sometimes push out other things via -security (such as the OpenSSL transition in bionic etc although one could argue that adds a security feature so is security related at least) so this is not without some precedent, however in this case we are thinking of using -security to workaround a bug / misfeature in older versions of snapd etc.