[WIP] Refresh App Awareness

I think I agree with you about failing the removal if there are apps running, but to be clear, my order of preference would be:

  1. Fail snap removal if there are running apps
  2. Kill running apps during snap removal
  3. Don’t kill running apps during snap removal

I hadn’t realized that 1 was an option, hence I was arguing for 2 instead of 3 which is the worst option to me (and also the current situation).

I had a look at visual studio code, hoping to see what is the method of detection of background snap refreshes. This is what I found https://github.com/microsoft/vscode/blob/597d8da84a8f5c7263aa9fbe90984b35807a1b27/src/vs/platform/update/electron-main/updateService.snap.ts#L201

In short, every now and then they look at the target of the current symlink. As such, with the current mechanism Visual Studio Code won’t be able to detect updates because there are none that would happen. I’m thinking if this warrants a discussion about the grander role of the current symlink and reliability. We might be able to actually not use the current symlink at all for anything and still change it so that applications that choose to. look at it will see the “change”.

At the same time I’d much rather introduce a snapctl API call to allow apps to ask as well as a notification mechanism (either new hook or a simpler method that apps can easily integrate with filesystem notification services).

We discussed where to take this feature from here and here are some quick notes:

New changes for v2 (from top to bottom as they arrive)

  • add snapctl refresh-available that instantly (without talking to the store) tells snaps that a refresh is pending [1w]
  • add new lock that inhibits application startup during refresh process [2w]
    • the lock needs to be safe from unrelated errors - bound to process, bound to ephemeral file system object
  • add cgroup-based app termination mechanics to snapd (20/80 approach, simple polling until cgroups v2 make it easy) [2w]
  • add new UX for command line and GUI apps that displays refresh progress while that lock is
    held or while current is gone, and we are attempting to start the app
    • cli just shows the changes via snap socket [2-3w]
    • snap run sends signal to the session agent to display the UI and waits in the back for the refresh of the app to change [2+w]
  • session agent UI response for the signal [2-3w] (1w with zenity)

Interesting things but not for v2

  • add snapctl refresh-and-rerun that apps can use to tell snapd to actively refresh them after the app terminates
1 Like

We need to think a bit more about this, we do want a variant at some point that also talks to the store. That means two commands or a careful use of options.

We discussed that we do need a way for the app to ask snapd to take the lock before the app terminates otherwise it might be restarted before we get a chance to take the lock/start the refresh.

Current thinking is to have snapctl refresh-available --offline, to be true to that when we have a pending update because of running apps we should download enough to be able to proceed indeed even if “offline”, that means logic similar to what we have done now for remodeling.

I have had experimental.refresh-app-awareness set to true for a few weeks now, and it works as intended, preventing snaps from updating while the applications are running.

However I have more than once observed an issue which I believe is related: since chromium is running pretty much all the time, it never gets a chance to refresh itself, except when I reboot my machine. When I do and there’s a new revision, it gets refreshed, but if after rebooting I run chromium too soon, I find that the profile directory that’s stored under $SNAP_USER_DATA is incomplete, as if launching the app had interrupted an ongoing copy operation. I can tell that because my profile directory weighs more than 1GB, and when that happens the current profile is much smaller (usually a few hundred MB).

The workaround is to close the app, delete the new profile directory, copy the old one with the new revision number, and launch the application again. This is not user-friendly though.

Is this a known issue?

1 Like

This is a known issue. As a part of the new design, though this is not implemented yet, the application will not be allowed to start during the refresh operation.

It looks like experimental.refresh-app-awareness isn’t working for me. I have set it some time ago (have rebooted several times after setting option). But chromium snap updates while it running causing loss of changes made until I restart snap.

Option is set:
% sudo snap get core experimental.refresh-app-awareness
true

% ls -la ~/snap/chromium/current
lrwxrwxrwx 1 baz baz 3 Nov 22 13:11 /home/baz/snap/chromium/current -> 949

“current” link was relinked at 13:11, while I was using old version of snap until 15:32

% ls -la ~/snap/chromium/937/.config/chromium/Default/Login\ Data
-rw------- 1 baz baz 589824 Nov 22 15:32 '/home/baz/snap/chromium/937/.config/chromium/Default/Login Data'

What version of snapd are you on? Can you please paste the output of snap version.

% snap version
snap 2.42.1
snapd 2.42.1
series 16
ubuntu 19.10
kernel 5.3.0-19-generic

How long was chrome open for? We built a safety system into the refresh postponing logic so that at some time the application will still refresh.

Chromium was running for about 20-30 days

This is expected then. The current logic forces a refresh after 7 days.

Those numbers are not final. We are also more likely to gracefully signal this to the application and users alike as the feature progresses.

Is this the same behavior for daemons/service as well ? It seems my daemon gets refreshed when running even with the “endure” keyword used, should I opt-in for this experimental feature ?

Daemons and services are stopped for refreshes and restarted afterwards. If you want to keep a service running despite a refresh then you must use the refresh-mode: endure, as documented on https://snapcraft.io/docs/snap-format - I also noticed it is not documented on https://snapcraft.io/docs/snapcraft-app-and-service-metadata – paging @degville for a suggested edit.

2 Likes

Is there an ETA for marking the feature stable and making it the default behaviour?

I keep getting incoming bug reports where users are bitten by the chromium snap being refreshed while running, leading to profile corruption (all marked as duplicates of bug #1616650), and I suggest users to enable refresh-app-awareness, but it would be nice if this was on by default.

7 Likes

I’m doing my best but at the moment part of the feature is under review and I cannot proceed.

1 Like

Hey @zyga-snapd, has there been any progress on this? The chromium snap continues to receive new bug reports of it being refreshed while running, and losing profile data as a consequence.

Can we expect to have the feature on by default in an upcoming release of snapd?

And what is the status of the work on notifying applications when a refresh is available/has been installed? I’m willing to have the chromium snap react to such notifications, and would gladly test the feature in the early stages of its implementation if it could help get it available to everyone.

1 Like

There’s slow progress. We are landing parts of the feature and shrinking working branches.

1 Like

This is quite painful and I have snap set core experimental.refresh-app-awareness=true. I run Chromium constantly. I keep it open for weeks. It’s incredibly stable on Ubuntu, except for these snap refreshes bringing it down. Every time it has crashed I go look and the snap was updated.

One thing that was very interesting was I happened to check and see there was snap refresh before it crashed. I checked the about dialog in Chromium and it showed there was a new version pending. I didn’t expect that unless you’ve patched it to be snap aware. But it didn’t show the usual update pending icon over the menu elipses in the main UI. If that were visible it would be fantastic because I could easily notice that and manually restart it when I’m ready before a crash.