[WIP] Refresh App Awareness

It looks like experimental.refresh-app-awareness isn’t working for me. I have set it some time ago (have rebooted several times after setting option). But chromium snap updates while it running causing loss of changes made until I restart snap.

Option is set:
% sudo snap get core experimental.refresh-app-awareness
true

% ls -la ~/snap/chromium/current
lrwxrwxrwx 1 baz baz 3 Nov 22 13:11 /home/baz/snap/chromium/current -> 949

“current” link was relinked at 13:11, while I was using old version of snap until 15:32

% ls -la ~/snap/chromium/937/.config/chromium/Default/Login\ Data
-rw------- 1 baz baz 589824 Nov 22 15:32 '/home/baz/snap/chromium/937/.config/chromium/Default/Login Data'

What version of snapd are you on? Can you please paste the output of snap version.

% snap version
snap 2.42.1
snapd 2.42.1
series 16
ubuntu 19.10
kernel 5.3.0-19-generic

How long was chrome open for? We built a safety system into the refresh postponing logic so that at some time the application will still refresh.

Chromium was running for about 20-30 days

This is expected then. The current logic forces a refresh after 7 days.

Those numbers are not final. We are also more likely to gracefully signal this to the application and users alike as the feature progresses.

Is this the same behavior for daemons/service as well ? It seems my daemon gets refreshed when running even with the “endure” keyword used, should I opt-in for this experimental feature ?

Daemons and services are stopped for refreshes and restarted afterwards. If you want to keep a service running despite a refresh then you must use the refresh-mode: endure, as documented on https://snapcraft.io/docs/snap-format - I also noticed it is not documented on https://snapcraft.io/docs/snapcraft-app-and-service-metadata – paging @degville for a suggested edit.

2 Likes

Is there an ETA for marking the feature stable and making it the default behaviour?

I keep getting incoming bug reports where users are bitten by the chromium snap being refreshed while running, leading to profile corruption (all marked as duplicates of bug #1616650), and I suggest users to enable refresh-app-awareness, but it would be nice if this was on by default.

7 Likes

I’m doing my best but at the moment part of the feature is under review and I cannot proceed.

1 Like

Hey @zyga-snapd, has there been any progress on this? The chromium snap continues to receive new bug reports of it being refreshed while running, and losing profile data as a consequence.

Can we expect to have the feature on by default in an upcoming release of snapd?

And what is the status of the work on notifying applications when a refresh is available/has been installed? I’m willing to have the chromium snap react to such notifications, and would gladly test the feature in the early stages of its implementation if it could help get it available to everyone.

1 Like

There’s slow progress. We are landing parts of the feature and shrinking working branches.

1 Like

This is quite painful and I have snap set core experimental.refresh-app-awareness=true. I run Chromium constantly. I keep it open for weeks. It’s incredibly stable on Ubuntu, except for these snap refreshes bringing it down. Every time it has crashed I go look and the snap was updated.

One thing that was very interesting was I happened to check and see there was snap refresh before it crashed. I checked the about dialog in Chromium and it showed there was a new version pending. I didn’t expect that unless you’ve patched it to be snap aware. But it didn’t show the usual update pending icon over the menu elipses in the main UI. If that were visible it would be fantastic because I could easily notice that and manually restart it when I’m ready before a crash.

If helpful: Today I noticed (disk light flashing) during my snap Chromium [edit: No experimental flags anywhere] session that it was auto-refreshing. Soon there was a KDE(?) notification (paraphrased) “Chromium has updated and needs to restart”.

I had made a noteworthy bookmark in this session prior to this update.

I closed and restarted. :white_check_mark:The noteworthy bookmark was still there (this has failed in the past) and :white_check_mark:Chromium did not issue a “did not close properly” warning upon restart (which had been typical in this scenario).

It’s possible I am misreporting the facts but I did pay attention because I was concerned I was going to lose my new bookmark.

I’m glad the notification served its purpose :slight_smile: . Typically, the bookmark would have been lost if you had created it after the refresh to the new version.

Hello everyone. I’d like to post a small update to aid discussing another chunk of work related to this feature.

Context

Currently snapd performs two checks, dubbed the soft and hard checks, as it performs a refresh of a particular snap. Both checks look at the processes associated with the snap being refreshed but differ in how they decide if the check succeeded or not.

The soft check fails if there are any applications or hooks running. Services are ignored as they are normally stopped later in the refresh process. This check catches the simple instance of an application being open.

The hard check is more strict as it runs after services have been stopped. At this time presence of any application process is a problem as snapd is about to create a copy of the snap data directory and the running application can affect the process.

That’s the current state, now for the new component.

Run inhibition lock

I’d like to introduce a new mechanism, called the run inhibition lock. The lock is scoped to a particular snap and is persistent across reboots. Once held the lock causes snap run to pause and wait for the lock to be released. While waiting the user is notified that a snap is being refreshed with either a graphical information dialog or directly in the console (if a tty exists).

The lock file is /var/lib/snapd/inhibit/$SNAP_NAME.lock. Unlike the snap lock, which is stored in /run/snapd/lock/$SNAP_NAME.lock, the file may exist for arbitrary amount of time The snap lock is ephemeral and has a hard limit of being locked for several seconds only.

The lock files are created and removed by snapd and read by snap run.

Hint

The inhibition lock may be empty or may contain a hint. The hint exist to communicate, without having to talk to snapd, why the lock exists. Currently the only value that is defined is the word refresh, which indicates that snapd is performing a refresh operation that affects this snap.

Empty hint indicates that snap execution is uninhibited.

Non-empty hint indicates that snap execution is inhibited.

Expected usage

I wrote a prototype that grabs the run inhibition lock during the snap-unlink task and removes it during the snap-link task. This is technically correct, as the snap cannot be used during that window. I’d like to expand the lock region to start immediately during the soft check mentioned above. While the snap lock is still held. This would allow snapd to detect that a refresh is possible and that it will be possible by the time the hard check is performed.

Inhibition lock operations

The lock files are stored in /var/lib/snapd/inhibit/$SNAP_NAME.lock.

LockAndSetHint

  • open the lock file as 0644
  • obtain an exclusive lock (LOCK_EX)
  • write the hint content
  • close the file.

Unlock

  • open the lock file
  • obtain an exclusive lock
  • truncate the file
  • close the file

IsLocked

  • open the lock file - if missing return informing the lock is not held
  • obtain a shared lock (LOCK_SH)
  • stat the file - if empty then close and return informing the lock is not held
  • read the contents and return informing the lock is held and the hint is the read value
  • close the file
3 Likes

I’ve implemented this as https://github.com/snapcore/snapd/pull/8566

I’ve updated the simple UI branch to use the mechanism outlined above: https://github.com/snapcore/snapd/pull/7700

I’ve pushed many extra branches towards making this feature complete.

For a better overview please track https://github.com/snapcore/snapd/projects/3

only if the check succeed? will it removed later if the hard check fails?