Personal-files request for doctl [Was: Classic confinement Request: doctl]

jdstrand · July 9, 2019, 10:00pm

What I granted covers the case for $HOME/.config/doctl/.config since the apparmor rule that is used is owner "@{HOME}/.config/doctl{,/,/**}" rwkl,. Since you wanted write access, I intentionally did not grant write access to $HOME/.config or $HOME/.kube (note the kube-config access I granted is known to work elsewhere). Can you try with what I suggested and report back if you have security policy denials?

hilz · July 11, 2019, 10:26pm

@jdstrand TLDR; doctl-config looks good, but kube-config isn’t working.

In depth: I tested your version on a DO droplet.

What I did:

Build and installed doctl snap locally (--dangerous) under strict confinement.

Tested doctl-config:

$ doctl auth init
DigitalOcean access token: 
Validating token... OK

initialize configuration

Usage:
    <snip>
Error: open /home/hilary/.config/doctl/config.yaml: permission denied
$ sudo snap connect doctl:doctl-config
$ doctl auth init
DigitalOcean access token:
Validating token... OK

Assuming that some magic happens at the time of review that enables auto-connection, doctl-config looks good.

Tested kube-config:

$ doctl k8s cluster list
ID                                      Name     Region    Version        Auto Upgrade    Status     Node Pools
35005dfa-e10c-4bd7-bf3f-5b306f53483d    fred2    sfo2      1.14.1-do.3    false           running    fred2-default-pool
$ doctl k8s cluster kubeconfig save fred2
save a cluster's credentials to your local kubeconfig

Usage:
  doctl kubernetes cluster kubeconfig save <cluster-id|cluster-name> [flags]

<snip>
Error: Error loading config file "/home/hilary/.kube/config": open /home/hilary/.kube/config: permission denied
$ sudo snap connect doctl:kube-config
$ doctl k8s cluster kubeconfig save fred2
Notice: adding cluster credentials to kubeconfig file found in "/home/hilary/.kube/config"
Notice: setting current-context to do-sfo2-fred2
save a cluster's credentials to your local kubeconfig

<snip>
Error: open /home/hilary/.kube/config.lock: permission denied

I rebuilt and installed locally in devmode (--dangerous --devmode) and tried again.

When a snap is in devmode, runtime confinement violations will be allowed but reported. These can be reviewed by running journalctl -xe.

Unfortunately, running in devmode did not report any runtime confinement violations that I could find using journalctl (I tried using journalctl -x --follow when journalctl -xe did not work. The snap behaved as if it were in classic confinement, no warnings, etc.

jdstrand · July 12, 2019, 2:40pm

When you install with --dangerous, there are no signed assertions (ie, no snap declaration to apply auto-connection), so this is expected.

jdstrand · July 12, 2019, 2:46pm

Please do: sudo snap install snappy-debug. Thin in one terminal, do: sudo snappy-debug.

Then in another terminal, install doctl with --dangerous, run snap connect doctl:kube-config and then run doctl k8s cluster kubeconfig save fred2 (or whatever). Please paste any output from snappy-debug.

lucyllewy · July 12, 2019, 3:34pm

The way microk8s does the access to .kubeconfig is to bundle a copy of kubectl within and allows use of that via a namespaced name: microk8s.kubectl. in that way it doesn’t require read nor write access to ~/.kube/config because kubectl is local to the snap and thus knows about the snap-specific $HOME ($SNAP_USER_DATA).

hilz · July 15, 2019, 1:51am

TLDR; I list the error I got using the plugs config you gave. I then do a bit more diagnostics to help figure out what’s missing.

I followed your directions and snappy-debug spit out an error:

$ sudo snappy-debug
[sudo] password for hilary:
INFO: Following '/var/log/syslog'. If have dropped messages, use:
INFO: $ sudo journalctl --output=short --follow --all | sudo snappy-debug
kernel.printk_ratelimit = 0
= AppArmor =
Time: Jul 15 01:41:54
Log: apparmor="DENIED" operation="mknod" profile="snap.doctl.doctl" name="/home/hilary/.kube/config.lock" pid=24755 comm="doctl.real" requested_mask="c" denied_mask="c" fsuid=1000 ouid=1000
File: /home/hilary/.kube/config.lock (write)
Suggestions:
* adjust program to write to $SNAP_DATA, $SNAP_COMMON, $SNAP_USER_DATA or $SNAP_USER_COMMON
* add 'personal-files (see https://forum.snapcraft.io/t/the-personal-files-interface for acceptance criteria)' to 'plugs'

Hmmm. Both personal-files plugs show as manually connected when I list the connections:

$ snap connections doctl
Interface       Plug                Slot             Notes
home            doctl:home          :home            -
network         doctl:network       :network         -
personal-files  doctl:doctl-config  :personal-files  manual
personal-files  doctl:kube-config   :personal-files  manual
ssh-keys        doctl:ssh-keys      -                -

FYI, if I add a line to the kube-config plug in snap/snapcraft.yaml as follows, it works

plugs:
  <snip>
  kube-config:
    interface: personal-files
    write:
      - $HOME/.kube
      - $HOME/.kube/config

Thinking about it, that makes sense, as the error I’m getting is that a permissions error creating the lock file (permissions on the directory). I’m not getting that error for doctl-config because my configuration for that plug is for the directory, not the specific config file.

If I’m reading the tea leaves correctly, I need to write to both $HOME/.kube/config and $HOME/.kube/config.lock.

Trying specifying $HOME/.kube/config.lock explicitly in snap/snapcraft.yaml, rather than the directory.

Ahhhh… the below version works

plugs:
  doctl-config:
    interface: personal-files
    write:
    - $HOME/.config/doctl
  kube-config:
    interface: personal-files
    write:
      - $HOME/.kube/config.lock
      - $HOME/.kube/config

hilz · July 15, 2019, 1:55am

Thanks for the info @lucyllewy. That approach seems eminently reasonable for microk8s. Unfortunately, I’m loathe to bundle kubectl with doctl in order to solve the snap issue, as k8s is just one of many types of resources that doctl interacts with. That way madness lies…

jdstrand · July 23, 2019, 3:21pm

I’ve adjusted the snap declaration to include config.lock.

hilz · September 23, 2019, 9:24pm

Hi @jdstrand,

We’ve had bug reports from users about the permissions denied error thrown when the $HOME/.config/doctl directory doesn’t exist (or, occasionally, $HOME/.config itself doesn’t exist). Could you please update the snap declaration to give us permission to create (at least) $HOME/.config/doctl and preferably $HOME/.config itself?

Thanks,
Hilary

jdstrand · September 25, 2019, 7:58pm

doctl is not the clear owner of the ~/.config directory. We don’t allow writes to this directory since snaps would be able to DoS session applications or loosen permissions. Also, allowing it in the personal-files interface would allow access to configuration of many applications outside of the snap.

-1 to add $HOME/.config to personal-files for doctl

This directory is typically created upon session start, so it is odd that you are seeing this. Perhaps your snap could try to use ~/.config/doctl and if ~/.config doesn’t exist, fallback to $SNAP_USER_DATA/.config/doctl (or similar)?

hilz · October 11, 2019, 5:44pm

What about letting us create $HOME/.config/doctl, which is the more common issue. AppArmor should let you add that specific permission: create the directory, rather than write to $HOME/.config more generally.

jdstrand · October 24, 2019, 2:48pm

This is acceptable and currently granted via the snap declaration in the store.