Python multiprocessing "permission denied" in strictly confined snap

snapcraft
1

When running Black within my strictly confined snap inside a multipass VM I get a permission denied error:

ubuntu@dotrun:/home/robin/Projects/ubuntu.com$ black webapp
Traceback (most recent call last):
  File "/home/robin/Projects/ubuntu.com/.venv/bin/black", line 8, in <module>
    sys.exit(patched_main())
  File "/home/robin/Projects/ubuntu.com/.venv/lib/python3.6/site-packages/black.py", line 4135, in patched_main
    main()
  File "/home/robin/Projects/ubuntu.com/.venv/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/robin/Projects/ubuntu.com/.venv/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/robin/Projects/ubuntu.com/.venv/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/robin/Projects/ubuntu.com/.venv/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/robin/Projects/ubuntu.com/.venv/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/robin/Projects/ubuntu.com/.venv/lib/python3.6/site-packages/black.py", line 464, in main
    sources=sources, fast=fast, write_back=write_back, mode=mode, report=report
  File "/home/robin/Projects/ubuntu.com/.venv/lib/python3.6/site-packages/black.py", line 529, in reformat_many
    executor = ProcessPoolExecutor(max_workers=worker_count)
  File "/snap/dotrun/15/usr/lib/python3.6/concurrent/futures/process.py", line 402, in __init__
    EXTRA_QUEUED_CALLS)
  File "/snap/dotrun/15/usr/lib/python3.6/multiprocessing/context.py", line 102, in Queue
    return Queue(maxsize, ctx=self.get_context())
  File "/snap/dotrun/15/usr/lib/python3.6/multiprocessing/queues.py", line 42, in __init__
    self._rlock = ctx.Lock()
  File "/snap/dotrun/15/usr/lib/python3.6/multiprocessing/context.py", line 67, in Lock
    return Lock(ctx=self.get_context())
  File "/snap/dotrun/15/usr/lib/python3.6/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/snap/dotrun/15/usr/lib/python3.6/multiprocessing/synchronize.py", line 59, in __init__
    unlink_now)
PermissionError: [Errno 13] Permission denied

This works if I run it directly in the multipass VM itself.

I suspect this error is similar to the one mentioned here, to do with access to /dev/shm:

https://stackoverflow.com/questions/2009278/python-multiprocessing-permission-denied

Has anyone else encountered this problem? Any idea how I could work around it?

2

Snapd places restrictions on the names of semaphores to prevent different snaps from interfering with each other. From the look of it, the same issue came up with people trying to use multiprocessing with the MacOS X sandbox:

https://bugs.python.org/issue19478

It looks like there is a configuration variable that controls the prefix for semaphores, but there is no public API to change it. One of the Python developers suggests accessing one of the private variables in a comment on that bug, and it looks like the same should work with current Python releases. Youā€™d want to add something like the following before using the package:

import multiprocessing
multiprocessing.current_process()._config['semprefix'] = '/snap.dotrun.mp'

(assuming your snap is called ā€œdotrunā€ as the traceback seems to indicate).

3

I tried this, and spent a while stepping through the Black and standard library code, but I canā€™t get it to work.

Iā€™ve confirmed that SemLock._make_name() is returning e.g. /snap.dotrun.mp-gq35e6mx, and yet still this is what happens in syslog:

Feb 25 09:14:11 rt480 kernel: [ 2778.115931] audit: type=1400 audit(1582622051.087:194): apparmor="DENIED" operation="open" profile="snap.dotrun.dotrun" name="/proc/29521/mounts" pid=29521 comm="python3" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000
Feb 25 09:14:11 rt480 kernel: [ 2778.243534] audit: type=1400 audit(1582622051.215:195): apparmor="DENIED" operation="mknod" profile="snap.dotrun.dotrun" name="/dev/shm/4afan5" pid=29521 comm="python3" requested_mask="c" denied_mask="c" fsuid=1000 ouid=1000

Is this related to this one? Python multiprocessing sem_open blocked in strict mode

I didnā€™t quite understand how @jdstrand appeared to get that working.

4

This does look like that other issue (note the mounts denial can likely be ignored as it is usually just noise. You can also connect mount-observe for that).

For getting just my code to work until https://github.com/sergiusens/snapcraft-preload/pull/29 is merged, see https://git.launchpad.net/~jdstrand/+git/test-sem-open/tree/snapcraft.yaml and the other files in the tree.

1 Like
5

Thanks @jdstrand that works a charm. Iā€™ve added the fix to my snap:

6

note, also, that the PR mentioned above was merged a short while ago, so you can use the snapcraft-preload library directly now.