Possibly missing apparmor network permission

What is the full list of interfaces you are using with this snap (minus greengrass-support)?

Currently…

$ snap connections bandwhich 
Interface           Plug                          Slot              Notes
greengrass-support  bandwhich:greengrass-support  -                 -
network             bandwhich:network             :network          -
network-control     bandwhich:network-control     :network-control  manual
network-observe     bandwhich:network-observe     :network-observe  manual
process-control     bandwhich:process-control     :process-control  manual
system-observe      bandwhich:system-observe      :system-observe   manual

What’s the output of cat /sys/fs/cgroup/devices/snap.<snap-name>.<snap-app>/devices.list ?

$ cat /sys/fs/cgroup/devices/snap.bandwhich.bandwhich/devices.list 
c 1:3 rwm
c 1:7 rwm
c 1:5 rwm
c 1:8 rwm
c 1:9 rwm
c 5:0 rwm
c 5:1 rwm
c 5:2 rwm
c 136:* rwm
c 137:* rwm
c 138:* rwm
c 139:* rwm
c 140:* rwm
c 141:* rwm
c 142:* rwm
c 143:* rwm
c 10:239 rwm
c 10:242 rwm
c 10:200 rwm

So try adding the capability snippet from before, re-load the apparmor profile, then run the program, and while the program is running, try doing this:

sudo rmdir /sys/fs/cgroup/devices/snap.bandwhich.bandwhich/devices.list

and see if the application starts working again.

rm: cannot remove '/sys/fs/cgroup/devices/snap.bandwhich.bandwhich/devices.list': Operation not permitted

Oh sorry typo on my part (leave out the device.list):

sudo rmdir /sys/fs/cgroup/devices/snap.bandwhich.bandwhich

It’s still not right.

I am seeing processes owned by root in the process list (like sshd) but not my own chrome browser (which is using bandwidth as I am on a call. Chrome (and other user processes) show up in the unconfined bandwhich. The dns lookups also don’t work.

Could you post your snapcraft.yaml link so I can debug a bit myself?

name: bandwhich
base: core18 # the base snap is the execution environment for this snap
adopt-info: bandwhich
summary: Single-line elevator pitch for your amazing snap # 79 char long summary
description: |
  Terminal bandwidth utilization tool (formerly known as "what")

architectures:
  - build-on: i386
  - build-on: amd64
  - build-on: s390x
  - build-on: arm64
  - build-on: ppc64el

grade: stable
confinement: strict

parts:
  bandwhich:
    plugin: rust
    source: https://github.com/imsnif/bandwhich.git
    source-tag: '0.9.0'
    build-packages:
      - build-essential
    override-pull: |
      snapcraftctl pull
      snapcraftctl set-version $(git describe --tags --abbrev=0)

apps:
  bandwhich:
    command: bandwhich
    plugs:
      - network
      - network-observe
      - system-observe
      - network-control
      - process-control
      - greengrass-support

I think creating a socket is a different operation from creating a raw socket. It feels like network should not allow you to create raw sockets. Perhaps a new interface or an option on the network interface.

CC @pedronis and @jdstrand for design ideas.

@zyga note that network and network-bind do not allow you to create a raw socket, but network-control does. IMHO I think network-control is sufficient for this task.

I would argue that network-control is too liberal. There’s more chance of @reviewers approving a fine-grained interface (configured variant of network or network-bind) than network-control when the only reason is to allow raw sockets. AFAIUI the plan is to move to very fine-grained interfaces which is why we have some interfaces marked as transitional because at the moment they grant more access than we really want to provide in the future:

Transitional interfaces are used by trusted snaps to access traditional Linux desktop environments that were not designed to integrate with snap isolation. As such, they will become deprecated as replacement or modified technologies that enforce strong application isolation become available.

c.f. https://snapcraft.io/docs/supported-interfaces

Perhaps network-control is too broad of an interface for something like this specifically, but I also don’t know of many applications that require access to just raw sockets without also needing other things. In fact for this application, just that raw socket access isn’t enough to get it to work, the application also needs something from the docker-support or greengrass-support interfaces to work properly (I haven’t figured out what yet though).

1 Like

@popey So regarding the original question about bandwhich, I have determined that what this application needs to run (in addition to plugging network-control) is the sys_ptrace capability, which is normally denied to all snaps, unless that snap plugs an interface which explicitly uses ptrace, such as greengrass-support or docker-support. I confirmed this by plugging:

$ snap connections bandwhich 
Interface        Plug                       Slot              Notes
network          bandwhich:network          :network          -
network-control  bandwhich:network-control  :network-control  manual
network-observe  bandwhich:network-observe  :network-observe  manual
process-control  bandwhich:process-control  :process-control  manual
system-observe   bandwhich:system-observe   :system-observe   manual

and then removing the line in /var/lib/snapd/apparmor/profiles/snap.bandwhich/bandwhich:

deny capability sys_ptrace,

I’m not sure what the best way to move forward with this is, but IMHO, I think we have the following options:

  • make process-control also an interface that declares it uses PtraceTrace() in the interface code in snapd so that denying the capability sys_ptrace snippet is not included in the policy
  • add a new interface (strawman name ptrace-control) which just sets that it controls ptrace and thus would also declare it uses PtraceTrace() in the interface code

@jdstrand (when you’re back) thoughts?

1 Like

A side-note I would like to say is that debugging why this snap was not working under strict (or devmode) confinement is difficult because the standard thing we recommend is to “install in devmode and note any denials/allows”. That wasn’t helpful because the thing that this application needs was already denied in the policy, so apparmor didn’t log it.

I would like to propose that for specifically only devmode snaps, in snapd we strip deny ... messages so that snaps in devmode are:

  1. more likely to work since denials in AppArmor policy are not put into complain mode for some reason
  2. easier to debug since if something is silenced with a deny rule in strict mode, in devmode now that will be an allow message and users will at least have something to look at

I might take a quick stab at a branch doing this… I opened a RFC PR for this proposal at https://github.com/snapcore/snapd/pull/8019

1 Like

process-control does not use ptrace though

We’ve thus far avoid this because ptrace trace to arbitrary processes would provide sandbox escape. What is the denial you are seeing?

Right I’m saying we add ptrace to process-control and then set that attribute

I do not see any additional denials. What happens is that when the snap is in devmode, with the following interfaces connected:

$ snap connections bandwhich 
Interface        Plug                       Slot              Notes
network          bandwhich:network          :network          -
network-control  bandwhich:network-control  :network-control  manual
network-observe  bandwhich:network-observe  :network-observe  manual
process-control  bandwhich:process-control  :process-control  manual
system-observe   bandwhich:system-observe   :system-observe   manual

there are these denials seen every few seconds:

Feb 27 11:13:21 kernel: audit: type=1400 audit(1582823601.779:1216): apparmor="ALLOWED" operation="capable" profile="snap.bandwhich.bandwhich" pid=85527 comm="display_handler" capability=2  capname="dac_read_search"
Feb 27 11:13:21 kernel: audit: type=1400 audit(1582823601.819:1217): apparmor="ALLOWED" operation="capable" info="optional: no audit" error=-1 profile="snap.bandwhich.bandwhich" pid=85527 comm="display_handler" capability=34  capname="syslog"

But if I comment out/remove this snippet from the generated apparmor profile:

deny ptrace (trace),
deny capability sys_ptrace,

Now the application works again, with no additional denials. If I had to guess what’s going on, I’d say that there are conflicting rules in the policy, namely that something is being allowed that falls under the broad ptrace (trace) spec, and we don’t see any denials or audits for those because of the deny rule, so when we remove that deny, now the application works because the other rule is allowed.

Also to be clear those denials happening above, happen after I remove the ptrace snippet as well.

If I disconnect all interfaces, and run in devmode, then I see these audits (sorted and de-duped but still ~200 lines): https://pastebin.ubuntu.com/p/MmvHsngqKh/ and the application does work correctly.

I took a look at this and with only network, network-observe and system-observe, these rules are needed:

network packet raw,          # only in network-control
capability sys_ptrace,       # also in network-control
capability dac_read_search,  # in home.go with read: all and system-backup

The network rule is needed to start the application. The dac_read_search rule seems to be needed to show non-root processes. I’m not sure why yet that the sys_ptrace rule is needed.

I can say that there appears to be a bug since if I connect network-control, this is still in the policy:

deny ptrace (trace),
deny capability sys_ptrace,

If I comment both of those out I only need to add the dac_read_search rule and it works. The ‘ptrace (trace),’ rule is not needed.

So, assuming that bug is fixed, the snap could:

plugs:
- network
- network-observe
- system-observe
- network-control
- process-control
- system-backup

and work. I need to spend some time with the snap to figure out the dac_read_search access and figure out how to get rid of system-backup.

Hi, I found I had to use system-backup for a simple snap [1] that uses snapd rest API (and if memory serves, this was not required a month or so ago). I tried network-control and network-bind, but even so I hit the dac_read_search denials. [2]

[1]

[2]

Time: Apr 16 15:34:19
Log: apparmor="DENIED" operation="capable" profile="snap.test-remodel.list" pid=29687 comm="list" capability=2  capname="dac_read_search"
Capability: dac_read_search
Suggestions:
* adjust program to not require 'CAP_DAC_READ_SEARCH' (see 'man 7 capabilities')
* add one of 'system-backup' to 'plugs'
* do nothing if program otherwise works properly```