Htop snap unable to signal aa-enforced processes

[maxiberta]

Hi,
htop snap gets apparmor="DENIED" when trying to SIGTERM kill firefox and chromium-browser renderer processes. Both /usr/lib/firefox/firefox and /usr/lib/chromium-browser/chromium-browser are in enforce mode according to aa-status. Setting these to complain mode fixes this issue. But, otoh, /usr/bin/htop is able to signal these processes successfully even in enforce mode.

Is it possible to make htop snap signal aa-enforced processes, as /usr/bin/htop does?

Note this happens with different versions of htop snap (I’m currently on candidate, rev 163, v2.2.0); and it is connected to :process-control and :system-observe, as expected.

Btw, I’m the current maintainer of this snap.

$ journalctl -fk
...
May 01 11:06:50 max5 kernel: audit: type=1400 audit(1525183610.173:770): apparmor="DENIED" operation="signal" profile="/usr/lib/firefox/firefox{,*[^s][^h]}" pid=32120 comm="htop" requested_mask="receive" denied_mask="receive" signal=term peer="snap.htop.htop"
May 01 11:06:50 max5 kernel: audit: type=1400 audit(1525183610.173:771): apparmor="DENIED" operation="signal" profile="/usr/lib/chromium-browser/chromium-browser" pid=32120 comm="htop" requested_mask="receive" denied_mask="receive" signal=term peer="snap.htop.htop"

$ snap version
snap    2.32.5+18.04
snapd   2.32.5+18.04
series  16
ubuntu  18.04
kernel  4.15.0-20-generic

Cheers,

[jdstrand]

Signal mediation requires that the sender be able to send the signal (to the receiver) and the receiver needs to be able to receive the signal (from the sender). While the htop profiles has rules that allow sending signals anywhere, the firefox and chromium profiles you listed do not have rules in them that allow htop to send signals to them. For these profiles in Ubuntu, you can work around this by adding the following to:

• /etc/apparmor.d/local/usr.bin.firefox: signal (receive) peer=snap.htop.htop,
• /etc/apparmor.d/local/usr.bin.chromium-browser: signal (receive) peer=snap.htop.htop,

Then run:

$ sudo apparmor_parser -r  /etc/apparmor.d/usr.bin.firefox
$ sudo apparmor_parser -r /etc/apparmor.d/usr.bin.chromium-browser

What is interesting though is that htop still wouldn’t be able to send signals to other snaps, since while the htop snap is allowed to send signals anywhere, strict mode snaps don’t have a corresponding rule to receive the signal from htop. @niemeyer, snaps not being able to receive signals from htop is an interesting problem, and one snapd could solve by:

for sender in <each snap command that plugs process-control>:
  for receiver in <all strict and devmode snaps>:
    add this rule to the receiver policy: ```send (receive) peer=snap.<sender>,```

(we don’t have to do classic because they already include a bare signal rule, but we could add the rule anyway to simplify the implementation).

system-observe is also affected by this with its ‘ptrace (read)’ rule (we would want to add ptrace (readby) peer=snap.<snap command plugging system-observe>,

@niemeyer - AFAIK, we don’t have a concept for something like the above though, or would/could this be implemented as a hook of some sort?

An alternative implementation to the above would be to maintain ‘.d’ style directories. We then add to the default template:

#include "/var/lib/snapd/apparmor/process-control
#include "/var/lib/snapd/apparmor/system-observe

Then on snap connect htop:process-control we create /var/lib/snapd/apparmor/process-control/snap.htop.htop with signal (receive) peer=snap.htop.htop,, then reload all the profiles. While the implementation is different, it has the exact same effect policy-wise as modifying each profile directly.

[maxiberta]

I think this might be related. The current candidate revision of htop now includes lsof to display the open files of processes, by pressing "l". But, as might be expected, confinement blocks the operation, sometimes with the following messages (haven’t found why sometimes it won’t produce any messages, but it always fails):

May 15 13:08:11 max5 kernel: audit: type=1400 audit(1526400491.395:829541): apparmor="DENIED" operation="open" profile="snap.htop.htop" name="/proc/3480/fdinfo/3" pid=3480 comm="lsof" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000
May 15 13:08:11 max5 kernel: audit: type=1400 audit(1526400491.395:829542): apparmor="DENIED" operation="open" profile="snap.htop.htop" name="/proc/3480/mounts" pid=3480 comm="lsof" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000

[jdstrand]

These are actually unrelated (they are simple file rules with no peer counterpart).

The first is not available in any interfaces. I’ve added a todo to add it to system-observe.

The other is provided by the ‘mount-observe’ interface.

[niemeyer]

@jdstrand The double declaration seems somewhat inconvenient, and it’s not clear to me that there are real advantages in doing that. Even if we go over the trouble of declaring both the sender and the receiver, the htop snap would still not be able to signal the external processes in enforce mode, right? And the external htop would also not be able to signal the snap processes, or would it?

From an admin point of view, that kind of blocking that renders traditional tools unable to terminate processes seems more like a misfeature than an actual protection.

Could we restructure the logic so the blocking happens on the sending end, so that snaps cannot send such signals unless they are connected, but everything else works correctly?

[jdstrand]

@niemeyer

@jdstrand The double declaration seems somewhat inconvenient, and it’s not clear to me that there are real advantages in doing that. Even if we go over the trouble of declaring both the sender and the receiver, the htop snap would still not be able to signal the external processes in enforce mode, right? And the external htop would also not be able to signal the snap processes, or would it?

The htop snap is able to send signals to everything because of its interface policy and unconfined (and its own, since snaps can signal themselves) and classic snap (since it has a bare signal rule) processes can receive those signals. Anything on the system that has an apparmor profile that is not unconfined (or htop itself) needs a corresponding receive rule to actually be signalled. This is inconvenient in this case, but this is the upstream kernel behavior and is useful in a number of situations.

However, we are in control of the policy and we have options:

1. add signal (receive) peer=snap.*, in the default template. We would want to investigate our existing policy to ensure this is safe, and adjust as needed. This is effectively what you suggested where we only mediate the sender
2. we manage the receiving end in a fine-grained in some manner (what I was initially exploring) such that snaps can only receive signals from other snaps if the sending snap uses process-control or is classic.

‘1’ has the benefit of being a simple policy change (provided all other signal rules are reviewed) but has the drawback that there is no fallback receive rule if there are policy bugs.

‘2’ has the benefit of being more defense in depth but at the cost of implementation complexity (though with the .d approach, it wouldn’t be terrible).

In thinking through this more, I think I’d like to explore ‘1’.

[niemeyer]

@jdstrand That’s in conflict:

@jdstrand

The htop snap is able to send signals to everything because of (…). Anything on the system that has an apparmor profile that is not unconfined (or htop itself) needs a corresponding receive rule to actually be signalled. (…)

Either it is able to signal everything or it can’t. What’s the case here? Being able to send to something that won’t receive is not useful.

@jdstrand

This is inconvenient in this case, but this is the upstream kernel behavior and is useful in a number of situations.

What situations?

@jdstrand

add signal (receive) peer=snap.*, in the default template. We would want to investigate our existing policy to ensure this is safe, and adjust as needed. This is effectively what you suggested where we only mediate the sender

That’s worth considering indeed, depending on the outcome of the discussion above.

[jdstrand]

I think it is important to note that in the case of IPC (signal, ptrace, etc) there are two processes involved which may be differently confined. In the general upstream case, it is correct for AppArmor to mediate the sender and receiver processes because AppArmor doesn’t dictate how policy is written and applied on the system.

It is perfectly valid for an administrator to write policy for a super-privileged daemon that says “this daemon should not receive signals from anywhere except for processes running under this admin-ui profile I created”.

It is also perfectly valid (if we decide this is the right approach, which I think we are leaning towards) for snapd to say “since I manage all sender snap rules already, let’s allow all snaps to receive signals from anywhere”. We just need to make sure that ‘since I mediate all senders already’ is accurate and all future policy takes that into account. (From a policy perspective, it is also perfectly valid to do the opposite: allow all sends and manage only receives, but that is pretty weird).

It is also perfectly valid for snapd to say “I’m going to manage both sender snap rules and receiver snap rules to provide defense in depth against profile errors”. This is the initial approach I initially discussed in this thread, but this comes at an implementation cost that I’m now thinking is not worth it.

In other words, the fact that AppArmor mediates sender and receiver isn’t a problem in and of itself, we just need to decide how to write the policy based on our requirements.

[niemeyer]

I see, sounds good, and thanks for the insightful explanation.

We can perhaps start with the easy and apparently quite nice solution of just opening it up as you suggest on (1), and then learn more about the problem. Since we are also managing senders, as you say, it won’t in principle be a problem.

[jdstrand]

@niemeyer

We can perhaps start with the easy and apparently quite nice solution of just opening it up as you suggest on (1), and then learn more about the problem. Since we are also managing senders, as you say, it won’t in principle be a problem

I’ve added this as part of my next batch of policy updates (though I think I’ll submit it in a separate PR for clarity).

[jdstrand]

https://github.com/snapcore/snapd/pull/5174

[jdstrand]

This is now merged in trunk.

[jdstrand]

@jdstrand

The first is not available in any interfaces. I’ve added a todo to add it to system-observe.

https://github.com/snapcore/snapd/pull/5189

[pachulo]

Hey @jdstrand thanks a lot for those changes, but it seems like there is still some work to do. When trying to list the open files of processes from the htop snap by pressing “l”, now I get this:

d’oct. 13 18:38:37 xps9360 kernel: audit: type=1400 audit(1539448717.077:424): apparmor="DENIED" operation="open" profile="snap.htop.htop" name="/proc/locks" pid=4782 comm="lsof" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0

I guess that the /proc/locks are not in any interface, are they?

[jdstrand]

@pachulo

I guess that the /proc/locks are not in any interface, are they?

They aren’t. I’ll add it to the next batch of policy updates (ie, to system-observe).

[maxiberta]

Curiously, pressing “l” (lsof) now seems to work on Ubuntu 18.10 only, while still failing on 16.04 and 18.04 (with all interfaces connected) with:

audit[17938]: AVC apparmor="DENIED" operation="open" profile="snap.htop.htop" name="/proc/locks" pid=17938 comm="lsof" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0
kernel: audit: type=1400 audit(1545235537.792:449): apparmor="DENIED" operation="open" profile="snap.htop.htop" name="/proc/locks" pid=17938 comm="lsof" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0

Works on 18.10:

snap    2.36.3
snapd   2.36.3
series  16
ubuntu  18.10
kernel  4.18.0-12-generic

Not working on 16.04:

snap    2.36.3
snapd   2.36.3
series  16
ubuntu  16.04
kernel  4.15.0-42-generic

Not working on 18.04:

snap    2.36.3
snapd   2.36.3
series  16
ubuntu  18.04
kernel  4.15.0-29-generic

[jdstrand]

@maxiberta

name="/proc/locks"

I’ve added this to the list for the next batch of updates.