Electron snap killed when using app.makeSingleInstance API

snapd
7

The /etc/xdg/user-dirs.conf denial is fixed in 2.29 (the core snap in the beta channel should have this fix).

The bluez denials are just noise and likely not the cause of your issues. Feel free to plug the bluez interface if you want them to go away.

The ptrace denial is almost certainly due to cascading failures (eg something earlier failed and mailspring is trying to trace it, etc). The ‘ptrace trace peer=unconfined’ is not allowed by any interfaces (it would be a huge security issue: you would be able to control any unconfined process (eg, systemd), dozens of processes on a classic system, etc). But, like I said, I doubt you need it.

The dconf denial is for mmap(). When I was looking at mailspring wrt interface auto-connections, I reported that the snap suffers from having an executable stack:

$ snap-review ./mailspring_42.snap 
Warnings
--------
 - functional-snap-v2:execstack
	Found files with executable stack. This adds PROT_EXEC to mmap(2) during mediation which may cause security denials. Either adjust your program to not require an executable stack, strip it with 'execstack --clear-execstack ...' or remove the affected file from your snap. Affected files: usr/share/mailspring/mailspring
	https://forum.snapcraft.io/t/snap-and-executable-stacks/1812
./mailspring_42.snap: FAIL

(snap-review is from the ‘review-tools’ snap in edge).

Please see Snap and executable stacks and adjust your snap accordingly.

The cause of your problems I think are the /run/user denials. Your snap is allowed to read things in /run/user/*/snap.mailspring/* that the user owns, but notice the denial has ‘fsuid=1000 ouid=0’. This indicates that your snap is creating /run/user/1000/snap.mailspring/.org.chromium.Chromium.7XwCRb/SS with root permissions and that your non-root user process is trying to access it. You mention this is an electron app: it sounds like you have the internal security sandbox turned on. Please specify --disable-sandbox or otherwise configure your snap to not use the internal sandbox and simply rely upon snapd’s sandbox. I am not an electron developer (but understand the issues surrounding having its sandbox turned on within a snap), but it looks like https://github.com/electron/electron/blob/master/docs/api/sandbox-option.md may be of some help.

In short, I think if you remove the executable stack and disable the internal sandbox, things should start to work for you (and optionally start using the beta core snap).

8

Hey @ogra, @jdstrand - thanks for the replies:

  • Adding the system-observe, bluez, and process-control interfaces removed some of the warnings from the syslog output but didn’t fix the issue. (As expected)

  • I can confirm that the /etc/xdg/user-dirs.confg denial is fixed in 2.29.

  • Thanks for the info about the exectuable stack. When you say you reported it, should I have been notified somehow? I didn’t receive any feedback about the review—didn’t realize there was a snap-review command. I’ll see if I can fix that.

  • I don’t think I’m enabling the Chromium renderer process sandboxing (Mailspring doesn’t need it) but I’ll double check. Is there any other way around this other than disabling the sandbox mode? For apps that /do/ need it, Chromium’s sandbox mode is pretty important and isn’t equivalent to the snapd sandbox—asking folks to disable it should never be a general solution to sandboxing Electron snaps. (Snap’s OS-level sandbox gives my app access to the user’s keychain, files in their home directory, etc. The Chromium renderer process sandbox ensures that windows running untrusted javascript don’t have access to any of that.)

9

You mean like gnome-keyring? I’m not qualified to debate the rest of your points, but that’s actually covered by the password-manager-service interface, which isn’t automatically connected.

10

Hey! Yeah that’s a good point—the default Snap sandbox is pretty limited. Mailspring stores your email passwords in your system keyring, so the snap uses the password-manager-service interface.

UPDATE:

I fixed the execstack issue by adding execstack --clear-execstack <executable> to the build workflow and used execstack -q /snap/mailspring/current/usr/share/mailspring/mailspring to verify that it was fixed in the installed snap, but I’m still seeing the issue. Looking into the /run/user denials next.

I’ve also found that the /run/user reads + denials only occur when Mailspring uses Electron’s app.makeSingleInstance API. If I remove the code below, the denials go away and multiple copies of Mailspring run side by side and neither is killed.

    const otherInstanceRunning = app.makeSingleInstance(commandLine => {
      const otherOpts = parseCommandLine(commandLine);
      global.application.handleLaunchOptions(otherOpts);
    });

makeSingleInstance uses Chromium’s ProcessSingleton, so my guess is something in this implementation is acting as root, though nothing looks out of place offhand. Will keep digging in to this. I could discontinue use of this Electron API, but I’d need to implement a replacement and theirs seems very complete.

https://chromium.googlesource.com/chromium/chromium/+/master/chrome/browser/process_singleton_linux.cc

11

It looks like the ProcessSingleton implementation actually kills the “other running instance” deliberately if it’s unable to communicate with it through a shared socket [code], so it definitely makes sense that those AppArmor denials (for the shared socket file) result in the new process killing the old process. It looks like I just need to figure out why / how that .org.chromium.Chromium.Dytsxb/SS file is being created with root permissions. If anyone has any insight based on that source file, I’d appreciate it! This is well outside of my knowledge area :slight_smile:

12

I tried to provide that insight. There are only two ways this should be happening:

  • you enabled the chromium setuid sandbox and so it is starting as root. When I did the snap-review tool, it didn’t complain about setuid binaries, so this probably isn’t it
  • you are using ‘daemon’ as one of your commands which creates the socket as root
13

Which issue? This only would fix the mmap (‘m’) denial I referenced.

14

This is covered here:

The allow-sandbox attribute for the browser-support interface allows using the internal sandbox, but gives extra privileges to the app and triggers a manual review. Electron apps typically can rely on the snapd sandbox (in fact, electron-builder is already building snap swithout the sandbox).

15

The review happened as part of the request for interface auto-connections, which happened on your behalf via the snap advocacy team. I thought the information was forwarded to you, but seems it wasn’t. That’s ok, the snap-review tool was the extent of my feedback wrt executable stack.

16

Oh sorry! Since you brought it up in this thread I thought the mmap denial might be related to this issue.

17

I tried to provide that insight. There are only two ways this should be happening:

you enabled the chromium setuid sandbox and so it is starting as root. When I did the snap-review tool, it didn’t complain about setuid binaries, so this probably isn’t it
you are using ‘daemon’ as one of your commands which creates the socket as root

Hey thanks for the reply!

  • I’m not using a daemon command. The snapcraft.yaml file is linked in my original post and only contains the app command.

  • I’ve verified that I am not enabling the Chromium sandbox. (It looks like --no-sandbox is the default now so passing it isn’t necessary [commit], and turning it on explicitly via mailspring --enable-sandbox causes the app to immediately crash. )

Since this bug is only exhibited when I use Electron’s app.makeSingleInstance API, I’ve updated the post title to reflect that and I’m going to keep digging in there.

I downloaded a few other Electron snaps and I’ve found the same problem occurs with mattermost-desktop, brave and wordpress-desktop. (Launch one and then run the launch command on a second command prompt. The running copy is killed and the new copy starts, which is not intended behavior.) I think it’s just more noticeable in Mailspring because the app is invoked frequently by mailto links. I imagine it will also prevent Brave from handling web links from other apps, though.

18

Thanks for this extra information. I haven’t yet had a chance to look further into this, but wanted to let you know I got the message.

19

Hey! Thanks for the reply—let me know if you get a chance to take a look and want me to try anything. Just for kicks, I tried packaging the snap with browser-support’s “allow-sandbox” option enabled, but it didn’t change the log output or fix the issue. (Not sure why it would, but I figured giving the app whatever extra permissions that entailed was worth a shot!)

20

I was able to reproduce this on Ubuntu 17.10 with gnome-shell/wayland. Importantly, it seems like an intermittent failure but truns out it is not (at least here). I launched mailspring from a terminal and it creates the socket with the correct permissions:

$ ls -ld /run/user/`id -u`/snap.mailspring/.org.chromium*
drwx------ 2 jamie jamie 80 Nov  8 10:11 /run/user/1000/snap.mailspring/.org.chromium.Chromium.g9eGF5

$ ls -l /run/user/`id -u`/snap.mailspring/.org.chromium*
total 0
lrwxrwxrwx 1 jamie jamie 19 Nov  8 10:11 SingletonCookie -> 3809248481036341864
srwxr-xr-x 1 jamie jamie  0 Nov  8 10:11 SS

If in a second terminal, without closing the first mailspring, when I launch another one from the command line, the first is not killed and the second says ‘Exiting because another instance of the app is already running.’. No apparmor denial for the socket. I tried this 30 times and it worked every time.

If I go to ‘File/Quit’ within mailspring, then the /run/user/1000/snap.mailspring/.org.chromium… is removed and cleaned up. In the first terminal I see the app is terminated. If in the first terminal I start the app and in a second try to start it, again it works fine (as described above) for 30 times.

However, I can reproduce the issue like so:

  • make sure the application is cleanly closed (eg, File/Quit)

  • in one terminal, launch mailspring. In this case, /run/user/1000/snap.mailspring/.org.chromium.Chromium.Aoy3tc is created and its socket has the correct permissions

  • click the ‘x’ button in the title bar (as opposed to File/Quit). The window goes away, but in the terminal we can see that the application is still running and the /run/user/1000/snap.mailspring/.org.chromium.Chromium.Aoy3tc is not cleaned up

  • launch mailspring in a second terminal. At this point, the mailspring in the first terminal exits, but a new socket directory is created: /run/user/1000/snap.mailspring/.org.chromium.Chromium.thlJJx and the first still exists. An apparmor denial is logged for the original socket directory:

    apparmor=“DENIED” operation=“file_perm” profile=“snap.mailspring.mailspring” name="/run/user/1000/snap.mailspring/.org.chromium.Chromium.Aoy3tc/SS" pid=17066 comm=“mailspring” requested_mask=“r” denied_mask=“r” fsuid=1000 ouid=0

but looking at the socket after the fact, the file has the correct permissions;

 ls -ld /run/user/`id -u`/snap.mailspring/.org.chromium*
drwx------ 2 jamie jamie 80 Nov  8 10:19 /run/user/1000/snap.mailspring/.org.chromium.Chromium.Aoy3tc
drwx------ 2 jamie jamie 80 Nov  8 10:20 /run/user/1000/snap.mailspring/.org.chromium.Chromium.thlJJx

$ ls -l /run/user/`id -u`/snap.mailspring/.org.chromium*
/run/user/1000/snap.mailspring/.org.chromium.Chromium.Aoy3tc:
total 0
lrwxrwxrwx 1 jamie jamie 19 Nov  8 10:19 SingletonCookie -> 8465438638122226111
srwxr-xr-x 1 jamie jamie  0 Nov  8 10:19 SS

/run/user/1000/snap.mailspring/.org.chromium.Chromium.thlJJx:
total 0
lrwxrwxrwx 1 jamie jamie 20 Nov  8 10:20 SingletonCookie -> 10029366826429995547
srwxr-xr-x 1 jamie jamie  0 Nov  8 10:20 SS

This appears to be a bug in mailspring/electron not actually closing and cleaning up properly when receiving a window close event and not being able to find what it requires and thus kills off the process. Not sure why the kernel is reporting the socket as being owned by uid ‘0’ though, this may be a kernel bug.

Interestingly, if I add this to the policy (ie, no owner match):

/run/user/[0-9]*/snap.@{SNAP_NAME}/{,.}org.chromium.*/SS r,

then launching mailspring in the second terminal after window close works in that it doesn’t kill the first process, we see ‘Exiting because another instance of the app is already running.’ and a mailspring window pops up.

While I would argue that mailspring/electron is not doing the right thing on a window close event, I am going to add a workaround rule to the browser-support policy for the non-owner SS read. Will try to find a simplified reproducer to explore chasing down the kernel bug.

1 Like
21

I filed https://launchpad.net/bugs/1731012 against apparmor for now (but note it isn’t clear this is an apparmor bug).

22

Hey @jdstrand thanks for the reply and the investigation.

Interestingly, if I add this to the policy (ie, no owner match):

/run/user/[0-9]/snap.@{SNAP_NAME}/{,.}org.chromium./SS r, then launching mailspring in the second terminal after window close works in that it doesn’t kill the first process, we see ‘Exiting because another instance of the app is already running.’ and a mailspring window pops up.

Ahh great! This totally makes sense. The code in that Chromium ProcessSingleton class I referenced earlier implements the business logic around these files, and they’re intentionally killing the first process if it can’t open the socket file (I think the idea is that it’s become unresponsive). Resolving the AppArmor denial and allowing access to that file should definitely prevent the process from being killed.

Thanks for adding a workaround rule— that seems like a good solution for now and I’ll see if I can create a small reproduction snap (though a minimal Electron app still isn’t super minimal…)

How long will it take for the policy change to reach the stable channel? I’d love to ship the Mailspring snap (I’m actually getting a couple emails / tweets a day about it because I said it was coming soon on the website :sweat_smile:) but I should probably wait until a fix for this is in the wild.

23

Not long :slight_smile:

https://github.com/snapcore/snapd/pull/4180 and https://github.com/snapcore/snapd/pull/4181

1 Like
24

More specifically, this will be in snapd 2.29.3 which I’m told people are targeting for Monday.

1 Like
25

Hey! I can confirm that this issue is fixed on my test machines after the 2.29.3 update. Thanks for the quick turnaround! Going to start directing users to download the snap instead of the deb / rpm packages in a few hours. :+1:

3 Likes
26

I’m getting this same issue now, but my snapd version is 2.56.2 (Electron is v19.0.14). If I run my app, it creates a folder /run/user/1000/snap.glint which looks like:

snap.glint
    ├─── dconf
    |    └─── user
    └─── scoped_dir0cJlEZ
         ├─── SingletonCookie -> 4779539152699396278
         └─── SingletonSocket

Did the names of these files/folders change from .org.chromium.*/SS to scoped_*/SingletonSocket and cause a regression here?

As far as I can make out, the workaround would be to adding this to the policy: /run/user/[0-9]*/snap.@{SNAP_NAME}/scoped_.*/SingletonSocket r, However I don’t really understand what that means or how to do that ^^

Maybe I should also mention that I am using core20 for snap (Electron v19 uses core18 by default, I had to upgrade it to solve a separate issue)