Missing capabilities in process-control & mount-observe?

[popey]

I’m trying to snap jellyfin, which is a media server, like emby and plex. I’m having difficulty getting it started. I’ve put it in a repo at https://github.com/popey/jellyfin-snap along with a handy script to build, install and run it. It fails to launch and I get the following errors.

2020-03-17T19:58:51Z systemd[1]: Stopped Service for snap application jellyfin.jellyfin.
2020-03-17T19:58:51Z systemd[1]: snap.jellyfin.jellyfin.service: Start request repeated too quickly.
2020-03-17T19:58:51Z systemd[1]: snap.jellyfin.jellyfin.service: Failed with result 'exit-code'.
2020-03-17T19:58:51Z systemd[1]: Failed to start Service for snap application jellyfin.jellyfin.

I get the following security issues:

= AppArmor =
Time: Mar 17 20:27:33
Log: apparmor="DENIED" operation="open" profile="snap.jellyfin.jellyfin" name="/sys/fs/cgroup/memory/system.slice/snap.jellyfin.jellyfin.service/memory.limit_in_bytes" pid=782954 comm="jellyfin" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
File: /sys/fs/cgroup/memory/system.slice/snap.jellyfin.jellyfin.service/memory.limit_in_bytes (read)
Suggestion:
* adjust program to not access '/sys/fs/cgroup/memory/system.slice/snap.jellyfin.jellyfin.service/memory.limit_in_bytes'

= AppArmor =
Time: Mar 17 20:27:35
Log: apparmor="DENIED" operation="file_lock" profile="snap.jellyfin.jellyfin" name="/proc/1/cgroup" pid=782954 comm="jellyfin" requested_mask="k" denied_mask="k" fsuid=0 ouid=0
File: /proc/1/cgroup (write)
Suggestion:
* adjust program to not access '@{PROC}/@{pid}/cgroup'

= AppArmor =
Time: Mar 17 20:27:35
Log: apparmor="DENIED" operation="file_lock" profile="snap.jellyfin.jellyfin" name="/sys/devices/virtual/net/lo/operstate" pid=782954 comm="jellyfin" requested_mask="k" denied_mask="k" fsuid=0 ouid=0
File: /sys/devices/virtual/net/lo/operstate (write)
Suggestion:
* adjust program to not access '/sys/devices/virtual/net/lo/operstate'

= AppArmor =
Time: Mar 17 20:27:35
Log: apparmor="DENIED" operation="file_lock" profile="snap.jellyfin.jellyfin" name="/sys/devices/virtual/net/lo/flags" pid=782954 comm="jellyfin" requested_mask="k" denied_mask="k" fsuid=0 ouid=0
File: /sys/devices/virtual/net/lo/flags (write)
Suggestion:
* adjust program to not access '/sys/devices/virtual/net/lo/flags'

= AppArmor =
Time: Mar 17 20:27:35
Log: apparmor="DENIED" operation="file_lock" profile="snap.jellyfin.jellyfin" name="/sys/devices/virtual/net/lo/speed" pid=782954 comm="jellyfin" requested_mask="k" denied_mask="k" fsuid=0 ouid=0
File: /sys/devices/virtual/net/lo/speed (write)
Suggestion:
* adjust program to not access '/sys/devices/virtual/net/lo/speed'

= AppArmor =
Time: Mar 17 20:27:35
Log: apparmor="DENIED" operation="file_lock" profile="snap.jellyfin.jellyfin" name="/run/systemd/resolve/stub-resolv.conf" pid=782954 comm="jellyfin" requested_mask="k" denied_mask="k" fsuid=0 ouid=102
File: /run/systemd/resolve/stub-resolv.conf (write)
Suggestions:
* adjust program to use $SNAP_DATA
* adjust program to use /run/shm/snap.$SNAP_NAME.*
* adjust program to use /run/snap.$SNAP_NAME.*
* adjust snap to use snap layouts (https://forum.snapcraft.io/t/snap-layouts/7207)

I have the following connections:

Interface        Plug                      Slot              Notes
desktop          jellyfin:desktop          :desktop          -
home             jellyfin:home             :home             -
home             jellyfin:home-all         :home             manual
mount-observe    jellyfin:mount-observe    :mount-observe    manual
network          jellyfin:network          :network          -
network-bind     jellyfin:network-bind     :network-bind     -
network-control  jellyfin:network-control  :network-control  manual
network-observe  jellyfin:network-observe  :network-observe  manual
opengl           jellyfin:opengl           :opengl           -
process-control  jellyfin:process-control  :process-control  manual
removable-media  jellyfin:removable-media  :removable-media  manual

Any ideas what I’m missing here?

[ijohnson]

@popey

File: /sys/fs/cgroup/memory/system.slice/snap.jellyfin.jellyfin.service/memory.limit_in_bytes (read)

@popey

File: /proc/1/cgroup (write)

Does jellyfin try to limit the amount of memory that some of it’s processes can use? Also, does it run any containers or use any kernel namespaces, etc. like that?

The other accesses are probably fine to add to an existing interface, but the cgroup ones are the ones that I think are the most problematic, as it seems like it’s trying to control itself in a powerful way we have only typically allowed for container snaps like docker, lxd, etc.

[popey]

I initially spoke to the upstream developers and it was their opinion this was internal .net stuff, not something they had implemented.

[lucyllewy]

I did have that working before they fixed the media scanner bug so I didn’t release it because that bug caused crashes… but it was working besides that. I’ll dig out my code and see what I did later…

[ijohnson]

FWIW, I did some cursory googling and it seems like the garbage collector in dot net tries to use these cgroup paths to do something (?), so maybe you could try looking into if there’s an env variable or something you could set to tell the garbage collector to turn off and just see if your snap runs then?

Also, does the snap work if you install it in devmode?

[lucyllewy]

turns out the problem with jellyfin was a ulimit on nofiles being too low. By default on Ubuntu the nofiles limit is set to 1024. Luckily snap confinement doesn’t prevent us bumping the limit to 4096, which seems to be sufficient to get jellyfin starting.

[jdstrand]

@popey

apparmor=“DENIED” operation=“open” profile=“snap.jellyfin.jellyfin” name="/sys/fs/cgroup/memory/system.slice/snap.jellyfin.jellyfin.service/memory.limit_in_bytes" pid=782954 comm=“jellyfin” requested_mask=“r” denied_mask=“r” fsuid=0 ouid=0

It probably makes sense to allow snaps to read their own cgroup information. Eg:

/sys/fs/cgroup/**/snap.@{SNAP_INSTANCE_NAME}.@{SNAP_COMMAND_NAME}.** rk,

This might be ok for the default template.

@popey

apparmor=“DENIED” operation=“file_lock” profile=“snap.jellyfin.jellyfin” name="/proc/1/cgroup" pid=782954 comm=“jellyfin” requested_mask=“k” denied_mask=“k” fsuid=0 ouid=0

We can add ‘k’ access to the default template.

@popey

apparmor=“DENIED” operation=“file_lock” profile=“snap.jellyfin.jellyfin” name="/sys/devices/virtual/net/lo/operstate" pid=782954 comm=“jellyfin” requested_mask=“k” denied_mask=“k” fsuid=0 ouid=0
apparmor=“DENIED” operation=“file_lock” profile=“snap.jellyfin.jellyfin” name="/sys/devices/virtual/net/lo/flags" pid=782954 comm=“jellyfin” requested_mask=“k” denied_mask=“k” fsuid=0 ouid=0
apparmor=“DENIED” operation=“file_lock” profile=“snap.jellyfin.jellyfin” name="/sys/devices/virtual/net/lo/speed" pid=782954 comm=“jellyfin” requested_mask=“k” denied_mask=“k” fsuid=0 ouid=0

We can add ‘k’ to interfaces with this access (eg, network-observe and network-control).

@popey

apparmor=“DENIED” operation=“file_lock” profile=“snap.jellyfin.jellyfin” name="/run/systemd/resolve/stub-resolv.conf" pid=782954 comm=“jellyfin” requested_mask=“k” denied_mask=“k” fsuid=0 ouid=102

We can add ‘k’ to the interfaces with access to this file (eg, various network* interfaces).

I’ve added these to the list for the next batch of policy updates.

[jdstrand]

@ijohnson

The other accesses are probably fine to add to an existing interface, but the cgroup ones are the ones that I think are the most problematic, as it seems like it’s trying to control itself in a powerful way we have only typically allowed for container snaps like docker, lxd, etc.

snappy-debug is being misleading, mapping these to ‘write’, but the access is for ‘k’ (lock). I don’t mind adding read lock access to existing interfaces. Based on your dot.net link, it looks like these are all read and that it will fail gracefully such that the accesses are noise.

[jdstrand]

@lucyllewy

turns out the problem with jellyfin was a ulimit on nofiles being too low. By default on Ubuntu the nofiles limit is set to 1024. Luckily snap confinement doesn’t prevent us bumping the limit to 4096, which seems to be sufficient to get jellyfin starting.

Are you saying that the denials go away with a proper limit? That would make some sense: the application might be failing and trying to figure out why for diagnostics, etc.

[jdstrand]

FYI, I added these accesses in https://github.com/snapcore/snapd/pull/8443

[lucyllewy]

Sorry, @jdstrand, I didn’t see this question.

I didn’t check whether the denials went away or not at the time, merely that Jellyfin was actually functional. The failure to launch Jellyfin was root-caused by the ulimit being too low, but I don’t know whether that also caused the accesses in @popey’s log that were denied by AppArmor. Raising the ulimit succeeds in getting Jellyfin operational.

With snapd version 2.44.2+20.04 and the revised ulimit I see only one denial when starting Jellyfin today (7 Apr 2020), but that might be as a result of your changes from PR#8443:

= AppArmor =
Time: Apr  7 18:39:01
Log: apparmor="DENIED" operation="open" profile="snap.jellyfin.jellyfin" name="/proc/698117/mountinfo" pid=698117 comm="jellyfin" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
File: /proc/698117/mountinfo (read)
Suggestions:
* adjust program to not access '@{PROC}/@{pid}/mountinfo'
* add 'mount-observe' to 'plugs'