Fluxctl personal-files [Was: Fluxctl snap wants to be classic]

fluxctl is part of https://github.com/weaveworks/flux - it’s a binary that lets you talks to a fluxd in your Kubernetes cluster, for this it uses k8s.io/client-go, just like kubectl does. This is the code that figures out where your kubeconfig lives and that’s not a deterministic list of files.

… and nice to see you around! :slight_smile:

Others have been using personal-files to access ~/.kube/config. Is what your snap needs different from that?

Maybe I’m doing it wrong. I’m stuck with this piece of yaml:

name: fluxctl
version-script: |
  FLUX_TAG="$(git tag -l | egrep -v '^(chart-|helm-|master-|pre-split)' | sort --version-sort | tail -n1)"
  if [ "$SNAPCRAFT_PROJECT_GRADE" = "stable" ]
  then
    echo "$FLUX_TAG"
  else
    GIT_REV="$(git rev-parse --short HEAD)"
    echo "$FLUX_TAG+$GIT_REV"
  fi
version: git
summary: fluxctl talks to Weave Flux and helps you deploy your code
description: |
  fluxctl talks to your Weave Flux instance and exposes all its
  functionality to an easy to use command line interface.
grade: devel # must be 'stable' to release into candidate/stable channels
confinement: strict
base: core18

plugs:
  personal-files:
    read:
      - $HOME/.kube/config
      - $HOME/.minikube/client.key
      - $HOME/.minikube/client.crt
      - $HOME/.minikube/ca.crt

parts:
  fluxctl:
    source: .
    plugin: go
    go-importpath: github.com/weaveworks/flux
    override-pull: |
      snapcraftctl pull
      export GOPATH=$(dirname $SNAPCRAFT_PART_INSTALL)/go
      export PATH=$GOPATH/bin:$PATH
      cd $GOPATH/src/github.com/weaveworks/flux
      dep ensure
    build-packages:
      - gcc
      - go-dep
    prime:
      - bin/fluxctl

apps:
  fluxctl:
    command: bin/fluxctl

@popey recommended I use the snapcraft snap and multipass (beta). I hope that’s good.

When trying to run the above, I get:

[daniel@reef bin ]$ fluxctl identity
Error: Could not load kubernetes configuration file: invalid configuration: no configuration has been provided
Run 'fluxctl identity --help' for usage.
[daniel@reef bin ]$ snap connections | grep flux
personal-files          fluxctl:personal-files                     :personal-files                 manual
[daniel@reef bin ]$

Unfortunately I can’t find out with strace, nor with snappy-debug.scanlog which read operation failed or which file was not found.

Help?

strace itself can be tricky (but can be made to work). Did you try snap run --strace[=<any optional strace args>] -- fluxctl identity?

1 Like

Great advice! Thanks, I did not know about this.

This is where I’m stuck now and I’m not sure if $HOME did not work in snapcraft.yaml, or if it’s a separate issue.

[pid 11850] newfstatat(AT_FDCWD, "/home/daniel/snap/fluxctl/x2/.kube/config", 0xc0000ff078, 0) = -1 ENOENT (No such file or directory)
[pid 11850] newfstatat(AT_FDCWD, "/home/daniel/snap/fluxctl/x2/.kube/.kubeconfig", 0xc0000ff148, 0) = -1 ENOENT (No such file or directory)
[pid 11850] openat(AT_FDCWD, "/home/daniel/snap/fluxctl/x2/.kube/config", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid 11850] newfstatat(AT_FDCWD, "/var/run/secrets/kubernetes.io/serviceaccount/token", 0xc0000ff218, 0) = -1 ENOENT (No such file or directory)
[pid 11850] write(2, "Error: Could not load kubernetes"..., 111Error: Could not load kubernetes configuration file: invalid configuration: no configuration has been provided
) = 111
[pid 11850] write(2, "Run 'fluxctl identity --help' fo"..., 41Run 'fluxctl identity --help' for usage.

When a snap starts, $HOME is set to ~/snap/<snap name>/<current snap revision> and fluxctl is apparently looking at $HOME (like a good snap should ;).

Since you actually want to know about the real home, you can adjust your snap accordingly. You can find the user’s home with: getent passwd $(id -un) | cut -d ':' -f 6.

@degville - as discussed - we should document this $HOME stuff…

2 Likes

Thanks @jdstrand again. This request can be closed. I’ve got it working (after quite a few round-trips). If anyone is going to look for example code in the future, it’s here:

https://github.com/weaveworks/flux/pull/1966

1 Like

Ah, apparently, it still needs review as it uses personal-files.

@dholbach - yes, it does. There is also an additional requirement that the interface reference needs to be something descriptive ( The personal-files interface)

Currently you have:

plugs:
  personal-files:
    read:
    - $HOME/.kube/config
    - $HOME/.minikube/client.key
    - $HOME/.minikube/client.crt
    - $HOME/.minikube/ca.crt

This should be chnaged to:

plugs:
  kube-config:
    interface: personal-files
    read:
    - $HOME/.kube/config
    - $HOME/.minikube/client.key
    - $HOME/.minikube/client.crt
    - $HOME/.minikube/ca.crt
apps:
  fluxctl:
    ...
    plugs:
    - kube-config
    ...

+1 for use and auto-connect of personal-files for the above accesses using the ‘kube-config’ interface reference.

@reviewers - can some of you also vote for this?

Reading through, personal-files for accessing ~/.kube and ~/.minikube makes sense to me. I’m +1 on the interface and auto-connection.

Thanks for the reviews and the feedback! I updated the PR and uploaded a new snap with the suggested changes.

2 votes for, 0 against. Granting use of personal-files as described in this topic.

FYI, I’ve approved r4 of the snap but you’ll need to release it to a channel. The snap will need to be manually approved until the review-tools change allowing the use of the kube-config interface reference is in production.

Since we last talked we moved on to https://github.com/fluxcd/flux/blob/master/snap/snapcraft.yaml#L50:L57

A colleague of mine wants to extend this and let me know that:

unfortunately, kind clusters can have --name set.
This means that while the files are under the same directory, they have arbitrary names. (~/.kube/kind-config-*)

Would this be possible in any way? Can we use *? Can we add directories to personal-files?

Paging @jdstrand and @mvo.

Yes. Eg:

plugs:
  dot-kube:
    interface: personal-files
    read:
    - $HOME/.kube

which would allow access to the $HOME/.kube file, $HOME/.kube directory and all of its contents.

1 Like

Super.

It took me a bit, but I uploaded a version with a bit more generalised permissions to the store now. It should be in the review queue.

I adjusted the snap declaration to allow the more generalized $HOME/.kube and $HOME/.minikube since a) this is read access and b) we’ve allowed this with other snaps that need access to ~/.kube. @reviewers - please comment if this is in error.

However, while your snap was granted subsets of ~/.kube and ~/.minikube before (so expanded is warranted, see above), it also added ~/.config/k3d, which is new and not discussed before. Can you describe what this access is for and why you need it?

Hey Jamie,

this came in from @stealthybox who added support for k3d (k3s in docker, where k3s is “k3s is the lightweight Kubernetes distribution by Rancher”) in this PR.

I realise this looks like it’s becoming a bit of a wild-growth of config files to look at, but it’s what we have to do to support a few varieties of Kubernetes flavours.

Thanks a lot in advance.

Have a great day,
Daniel