LXC: snaps don't update

No, not yet. I’ll try to come up with a solution soon but I’m focusing on release work and landing layout feature work. Still this is a high-priority bug so I’ll definitely work on this this week but I cannot yet give an estimate.

@zyga-snapd any progress here? It’s been a few weeks.

1 Like

I want to reiterate: no one using snaps in LXD will get more than three updates before they stop updating at all. Every snap. Including core. These are security updates, folks.

@zyga-snapd Can you please dump what you know about this issue here? No need to work further on it as I know you’re busy elsewhere… just dump what you know here and we’ll take it from there.

Ack, I’ll do that shortly.

@zyga-snapd You are saying that for a month. :slight_smile: Can you please just write down the notes here so we can get someone else to push it forward. Again, don’t feel bad for not doing it. It’s best to have you focused on pushing those tasks you have today than to be jumping back and forth here. If you actually don’t have any details, that’s okay too… just let us know so we have context to move on.

I reproduced this easily on 14.04 desktop and I started poking around trying to see how to resolve it but I haven’t nailed any particular solution. I don’t have more other than what stgraber originally reported on the LXD forum. My research was into one of two alternatives: prevent the problem from occurring by changing something early with a systemd job or by altering snap-confine to detect and correct the problem before running the first application process.

The problem is described here: https://discuss.linuxcontainers.org/t/snapd-cant-remove-old-revisions-when-running-inside-lxd/452/4

What is the problem? Can you please detail what you already know? This whole topic has absolutely no information.

As far as I understand when snaps are mounted in /snap/name/revision and /snap isn’t MS_SHARED we are getting affected because then all of those mounts are not shared and updates don’t propagate correctly

@zyga-snapd Thanks! Please go chase layouts!

@stgraber Any chance we might have your help fixing this issue in snapd? I know you have a lot of great insight in that area. Could it be simply a matter of remounting the filesystem in those specific cases instead of just overmounting it?

I was thinking about a Before=snapd.mount in our autogenerated mount units but I was under the impression that systemd must be already doing this (the before I mean). Perhaps something is wrong with the snapd.mount unit and the fallback code in snap-confine is doing the remount and we suffer the bug then. I didn’t check that.

Just blew another hour babysitting all my containers through updating all their snaps. It actually takes less time to blow away an entire testing container, create a new one, and set it all up for my daily tests, than it takes to update.

/me blows more time babysitting containers through another update…

Any progress here?

I’d be happy to help with this if I understood snap-confine at all. I’m afraid I don’t, so all I can really do is offer to help test. I lose hair in frustration every time an update comes out!

@niemeyer any progress on this?

@jdstrand you’re the only other person I know for sure has snap-confine experience. Do you have any insights, here?

I have a fix for this now. I’ll send a PR shortly, I’m trying to see if any tests breaks.

EDIT: I was fighting this all yesterday and it’s still not fixed. I’m trying another strategy. This is one nasty problem.

2 Likes

A quick update: @zyga-snapd put up a proposal that needs to be tweaked:

<zyga-solus> kyrofa: I discussed it with jdstrand, we need to tweak it slightly in order not to make snap-confine too powerful,
<zyga-solus> kyrofa: I’ll get back to it
<zyga-solus> kyrofa: I’m looking at why master breaks so often as this clamps our velocity a lot
<kyrofa> Ah okay, very good. Still a path forward, though?
<zyga-solus> yes, totally

Is anyone still around working on this problem? Or is everyone gone for holidays? I don’t know how to raise the importance of this. I’m seriously considering running stuff from source instead of using my own snaps, and that’s heartbreaking.

@niemeyer any chance you guys can chat about this during the sprint this week?