Ubuntu 16.04 host denies core18 snap python 3.6 library import

snapd
1

I’m trying to use some python 3 code in my configure hook.

So far pretty ugly, but here goes

passthrough:
  hooks:
    configure:
      environment:
        # make jsonmerge module available to the configure script
        PYTHONPATH: $PYTHONPATH:$SNAP/lib/python3.6/site-packages

parts:
  configure-hook-python-dependencies:
    plugin: python
    python-version: python3
    python-packages: [jsonmerge]

(side note: I use python3.6/site-packages because, from snapcraft --shell I can tell that this is where jsonmerge ends up. I don’t know how to make this work generically)

Later in my configure script i have a python3 -c.

/usr/bin/python3 -c '
from jsonmerge import merge
...
'

The configure script fails with

ImportError: /usr/lib/python3.6/lib-dynload/_csv.cpython-36m-x86_64-linux-gnu.so: failed to map segment from shared object

The corresponding apparmor denial is

AVC apparmor="DENIED" operation="file_mmap" profile="snap.x.hook.configure" name="/usr/lib/python3.6/lib-dynload/_csv.cpython-36m-x86_64-linux-gnu.so" pid=7910 comm="python3" requested_mask="m" denied_mask="m" fsuid=0 ouid=0

The apparmor profile, indeed, does not allow anything higher than python 3.5

$ apparmor_parser -p /var/lib/snapd/apparmor/profiles/snap.x.hook.configure | grep lib-dynload
  /usr/lib{,32,64}/python3.[0-5]/lib-dynload/*.so            mr,
  /usr/local/lib{,32,64}/python3.[0-5]/lib-dynload/*.so            mr,

So, In my understanding, the snap gets an entire python 3.6 runtime due to its using core18 as the base, and a pip install does happen for jsonmerge. But the 16.04 host apparmor python profile (which only knows pythong up to 3.5) prevent the snap from using 3.6+ shared libraries within that pip-installed package.

It feels intuitively wrong that the host restricts the view of the snap to the known versions of python at the time of release for the host OS. And the snap might be packaging python 3.7 for all we know, which not even core18 knows about. So either the stock ubuntu 16.04 is too restrictive, or the generation of the apparmor profile done in snapd is.

What is the correct way to fix this? I don’t want to change my host apparmor python abstraction profile to make it more permissive, because that step would have to be performed for everyone running 16.04, which also feels wrong. I’m looking for the snapcraft way to do this correctly, or a confirmation that this is a bug.

I tried to find where apparmor profiles are generated in the snapd repo, but came up empty. I could investigate further if someone can point out which part of the code builds the profile.

2

try changing this to

/usr/bin/env python3

so the interpreter from inside the snap gets used …

3

It should be possible to use python3 from your base. In the past we’ve had to tweak rules so different .so files were able to load, I guess it needs another tweak?

Hmm… nope, that’s not it. If I snap run --shell into a strictly-confined core18-based snap, I’m perfectly able to import _csv:

$ python3
Python 3.6.8 (default, Oct  7 2019, 12:59:55) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _csv
>>> _csv
<module '_csv' from '/usr/lib/python3.6/lib-dynload/_csv.cpython-36m-x86_64-linux-gnu.so'>
>>>

so there’s something else going on here.
Is your snap strictly confined?

4

@ogra it actually could be the converse: they might be shipping an extra python in their snap (why?!?), and that python might have a different ABI to the one in base whose libraries it’s picking up.

5

that seems wrong. Here’s what I have in a core18-based snap,

$ grep python /var/lib/snapd/apparmor/profiles/snap.icdiff.icdiff
  # for python apps/services
  #include <abstractions/python>
  /usr/bin/python{,2,2.[0-9]*,3,3.[0-9]*} ixr,
  deny /usr/lib/python3*/{,**/}__pycache__/ w,
  deny /usr/lib/python3*/{,**/}__pycache__/**.pyc.[0-9]* w,

maybe your snapd is old? What’s the output of snap version?

6

ohh… I see it now. It’s from abstractions/python. Hmm

@jdstrand how do we fix this?

7

I made the change you suggest, and get the same error. I think both pick the same interpreter, but of course your way is better.

8

$ snap version
snap 2.42.1
snapd 2.42.1
series 16
ubuntu 16.04
kernel 4.15.0-70-generic

9

Just a side note since I’ve been looking at this kind of thing, but I strongly recommend not doing this since this adds startup time to your snap. After all the python abstractions/apparmor stuff is sorted out your snap should know exactly which python it’s running and where. Using /usr/bin/env will try a whole set of different python locations (most of which don’t exist) before finding one that works. It may not matter much for your application, but in the name of performance I’d suggest hard-coding which python to use in your snap, specifically just for snaps.

10

We’ll make sure apparmor upstream is fixed (I think it already is) and add a rule to snapd for the interfaces that include that abstraction. I’ll be submitting a PR for this soon.

1 Like
11

I can confirm that issue, one of my snap worked perfectly fine on 19.10 but had same permission issues on Ubuntu 16.04, in that case I was trying to use python3 from the core18 snap

12

euh, so i was already doing the right thing from the beginning. noted.

13

Will 16.04 receive the fix through the xenial-updates repository?
I noticed that I had been using the xenial-security repository, so I was lagging
behind a bit. But looking at it now, even xenial-updates is lagging behind the
last release on github (2.42.2).

$ apt-cache madison snapd
     snapd |       2.40 | http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
     snapd | 2.37.4ubuntu0.1 | http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     snapd |      2.0.2 | http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
14

Yes. It should be in 2.43 and the snapd team will push that through Ubuntu’s standard stable updates process on their normal schedule. Note that on Ubuntu (it exists elsewhere as well for those distributions that choose to enable it) there is a thing call ‘reexec’ which allows snapd in the deb to reexec into a snapd provided by, for example, the core snap so that you can have a newer snapd sooner. If you ‘snap version’, you should see you are already using 2.42.2.

15

hmm i still have 2.42.1

16

2.42.2 is in the beta channel, see snap info snapd

you can:
sudo snap refresh --beta snapd
…to use it right now and later go back with:
sudo snap refresh --stable snapd
…once it is released

1 Like
17

ok i see it now, thanks

18

Can you indicate the commit id or pr link so i can tell when it makes it to the edge channel?

19

https://github.com/snapcore/snapd/pull/7779

1 Like