SELinux blocking snapd since update on Fedora 27

I have a Fedora 27 VM that I’ve been using to test snaps. I just woke it up, updated and rebooted it, and now snap operations freeze.

Applications launch (which previously launched okay), but I can’t install anything new. I checked the journal and it seems to be SELinux related.

snap    2.28.5-2.fc27
snapd   2.28.5-2.fc27
series  16
fedora  27
kernel  4.13.13-300.fc27.x86_64
Nov 30 16:13:30 localhost.localdomain setroubleshoot[2265]: SELinux is preventing snapd from getattr access on the lnk_file /etc/pki/tls/certs/ca-bundle.trust.crt. For complete SELinux messages run: sealert -l 82827efa-a12d-4ff2-932b-df008c1dd763
Nov 30 16:13:30 localhost.localdomain python3[2265]: SELinux is preventing snapd from getattr access on the lnk_file /etc/pki/tls/certs/ca-bundle.trust.crt.
                                                     
                                                     *****  Plugin catchall (100. confidence) suggests   **************************
                                                     
                                                     If you believe that snapd should be allowed getattr access on the ca-bundle.trust.crt lnk_file by default.
                                                     Then you should report this as a bug.
                                                     You can generate a local policy module to allow this access.
                                                     Do
                                                     allow this access for now by executing:
                                                     # ausearch -c 'snapd' --raw | audit2allow -M my-snapd
                                                     # semodule -X 300 -i my-snapd.pp
                                                     

Pulled the latest Fedora cloud image. Installed snapd and kernel-modules (to get squashfs driver). Things seem to be working just fine here:

[fedora@localhost ~]$ sudo snap install hello-world
hello-world 6.3 from 'canonical' installed
[fedora@localhost ~]$ snap list
Name         Version      Rev   Developer  Notes
core         16-2.29.4.2  3604  canonical  core
hello-world  6.3          27    canonical  -
[fedora@localhost ~]$ uname -a
Linux localhost.localdomain 4.14.5-300.fc27.x86_64 #1 SMP Mon Dec 11 16:00:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[fedora@localhost ~]$ snap version
snap    2.29.4-2.fc27
snapd   2.29.4-2.fc27
series  16
fedora  27
kernel  4.14.5-300.fc27.x86_64
[fedora@localhost ~]$ getenforce
Enforcing

getattr getting blocked by SELinux seems to be unrelated.

Before updating the system and snapd package can you check:

  • selinux is in enforcing mode
  • does it still freeze if selinux is in permissive?
  • long shot but check if there are unlabeled files find / -context '*unlabeled_t*', this should list only files under /var/lib/snapd/snap (the snaps that are installed and mounted)

If nothing unusual comes up, can you update the system to the latest kernel and snapd pacakge and try again?

Ok, woke up the vm.

  • getenforce reports Enforcing
  • There are no unlabeled files outside /var/lib/snapd/snaps.
  • Switched SELINUX to permissive, and tried to install a snap, now it downloads and installs the snap (I used hiri).
  • Switched back to enforcing, which of course re-breaks installing snaps.
  • Updated via dnf
  • Tried installing a snap, and get the same hanging process, and errors in journal

Would it help if I opened an ssh port to this and stuck your key on it so you can poke it directly?

After spending some time poking around @popey’s system, the issue is DBus/SELinux.

When the user does snap install foo, snapd will ask polkit if used user is authorized. This happens over DBus. The request is captured here:

‣ Type=method_call  Endian=l  Flags=0  Version=1  Priority=0 Cookie=20                                                                                                                                                                         
  Sender=:1.139  Destination=org.freedesktop.PolicyKit1  Path=/org/freedesktop/PolicyKit1/Authority  Interface=org.freedesktop.PolicyKit1.Authority  Member=CheckAuthorization                                                                 
  UniqueName=:1.139                                                                                                                                                                                                                            
  MESSAGE "(sa{sv})sa{ss}us" {                                                                                                                                                                                                                 
          STRUCT "sa{sv}" {                                                                                                                                                                                                                    
                  STRING "unix-process";                                                                                                                                                                                                       
                  ARRAY "{sv}" {                                                                                                                                                                                                               
                          DICT_ENTRY "sv" {                                                                                                                                                                                                    
                                  STRING "start-time";                                                                                                                                                                                         
                                  VARIANT "t" {                                                                                                                                                                                                
                                          UINT64 1262675;                                                                                                                                                                                      
                                  };                                                                                                                                                                                                           
                          };                                                                                                                                                                                                                   
                          DICT_ENTRY "sv" {                                                                                                                                                                                                    
                                  STRING "pid";                                                                                                                                                                                                
                                  VARIANT "u" {                                                                                                                                                                                                
                                          UINT32 8488;                                                                                                                                                                                         
                                  };                                                                                                                                                                                                           
                          };                                                                                                                                                                                                                   
                  };                                                                                                                                                                                                                           
          };                                                                                                                                                                                                                                   
          STRING "io.snapcraft.snapd.manage";                                                                                                                                                                                                  
          ARRAY "{ss}" {                                                                                                                                                                                                                       
          };                                                                                                                                                                                                                                   
          UINT32 1;                                                                                                                                                                                                                            
          STRING "";                                                                                                                                                                                                                           
  };         

However, the reply is blocked by DBus daemon and a message is sent back from dbus-daemon to polkitd:

        ‣ Type=error  Endian=l  Flags=1  Version=1  Priority=0 Cookie=341  ReplyCookie=384                                                                                                                                                             
  Sender=org.freedesktop.DBus  Destination=:1.332                                                                                                                                                                                              
  ErrorName=org.freedesktop.DBus.Error.AccessDenied  ErrorMessage="An SELinux policy prevents this sender from sending this message to this recipient, 0 matched rules; type="method_return", sender=":1.332" (uid=997 pid=7025 comm="/usr/lib$
polkit-1/polkitd --no-debug " label="system_u:system_r:policykit_t:s0") interface="(unset)" member="(unset)" error name="(unset)" requested_reply="0" destination=":1.139" (uid=0 pid=3047 comm="/usr/libexec/snapd/snapd " label="system_u:sy$
tem_r:snappy_t:s0")"                                                                                                                                                                                                                           
  MESSAGE "s" {                                                                                                                                                                                                                                
          STRING "An SELinux policy prevents this sender from sending this message to this recipient, 0 matched rules; type="method_return", sender=":1.332" (uid=997 pid=7025 comm="/usr/lib/polkit-1/polkitd --no-debug " label="system_u:sy$
tem_r:policykit_t:s0") interface="(unset)" member="(unset)" error name="(unset)" requested_reply="0" destination=":1.139" (uid=0 pid=3047 comm="/usr/libexec/snapd/snapd " label="system_u:system_r:snappy_t:s0")";                            
  };

While this happens, the snap process is waiting for snapd, but the latter is blocked waiting for polkit to respond which will never happen.

Since the error message is sent only over DBus to polkitd it’s not really logged elsewhere. Also it does not appear in audit log, since the policy is applied by DBus.

On Monday, I’ll dig in and see if anything can be done with DBus policy files.

@jamesh i’m open to any suggestions you might have.

1 Like

Turns we were missing a policy to allow polkitd to send messages (in this case replies) back to snapd. PR that fixes the problem:

1 Like

snapd updates with the PR backported have been submitted:

1 Like

@Conan_Kudo thanks for taking case of package updates for Fedora.

I"ve bumped selinux policy version and pushed an update to the policy so that it is closed to what fedora-selinux uses. The policy is now:

# allow communication with polkit over dbus
optional_policy(`
  policykit_dbus_chat(snappy_t)
')

I think we will need to revisit SELinux policies. It’s permissive but we’ll probably want to switch it to enforcing at some point.

1 Like