+1 from me as well with snap using as described in @jdstrand reply above. The kube-config interface is limited to read only to $HOME/.kube for k9s and popeye.
@jdstrand @Igor - Thank you both for looking into it! I do have a question.: I’ve noticed that kubectl snap is using classic confinement. Given K9s and Popeye are is essence using the same configuration do you guys feel that I should be using this confinement instead and force users to specify --classic?
@jdstrand - I’ve just tried to push a new K9s snap based on your recommended snap config updates. Can you please verify my changes to make sure I did this correctly based on your recommendation?
Guessing I’ve missed something here based on the following message:
The Store automatic review failed.
A human will soon review your snap, but if you can’t wait please write in the snapcraft forum asking for the manual review explicitly.
If you need to disable confinement, please consider using devmode, but note that devmode revision will only be allowed to be released in edge and beta channels.
Please check the errors and some hints below:
- override not found for ‘plugs/kube-config’. Use of the personal-files interface is reserved for vetted publishers. If your snap legitimately requires this access, please make a request in the forum using the ‘store-requests’ category (https://forum.snapcraft.io/c/store-requests), or if you would prefer to keep this private, the ‘sensitive’ category.
- human review required due to ‘allow-installation’ constraint (bool)
2 votes for, 0 against for granting read-only access to $HOME/.kube with auto-connection for both k9s and pop;eye. This is now live.
@derailed - it didn’t pass automated review for two reasons: 1) the votes hadn’t been tallied and so the request wasn’t processed (I did that just now) and 2) the review-tools need a separate update for the use of ‘kube-config’ since we don’t yet have a way to grant that via snap declaration. I’ve made the change to the review-tools for this, but it isn’t in production yet so they’ll have to be manually approved until it is.
FYI, popeye should be converted too. I see you are trying to have popeye access ~/.k9s. I suggest perhaps instead reorganizing things or using a content interface for this. That isn’t strictly required though since you are the owner of both snaps.
@jdstrand Thank you so much for the update and for your time on this issue! I’ve just corrected and pushed Popeye v0.3.8 but it hangs on the publish since 0.3.0 was never approved and it’s stuck in the queue with no way for me to remove it. Not sure how to proceed here. Can you take a look?
I saw this in the store queue and responded there (you left out ‘interface: personal-files’ with your interface reference).
@jdstrand Thank you for enabling K9s and Popeye builds! It looks like the snap builds are coming thru now.
K9s snaps can’t still access $HOME/.kube for reading or $HOME/.k9s directory for read/write. Several users are reporting issues with the snap builds. K9s is not able to load/save configuration is $HOME/.k9s. For the .kube dir, setting $KUBECONFIG env var seems to work to locate the .kube dir but this less than ideal. K9s tells user their log files are in /tmp/k9s.log but in reality they land on /tmp/snap.k9s/tmp/… I think the same apply to the user home directory. So the question here what do I need to do to make this work as expected. I am hoping I don’t have to change my code to do anything special when released as a snap. Is this correct?
So at this time, K9s can’t read for $HOME/.kube or read/write to $HOME/.k9s. I am not sure how to best proceed from here… Please advise. Thank you!
For $HOME/.kube, it looks like the snap has everything it needs:
$ sudo snap install k9s
k9s 0.7.9 from Fernand Galiana (derailed) installed
$ snap list k9s
Name Version Rev Tracking Publisher Notes
k9s 0.7.9 138 stable derailed -
$ snap connections k9s
Interface Plug Slot Notes
home k9s:home :home -
network k9s:network :network -
personal-files k9s:kube-config :personal-files -
and from the apparmor policy in /var/lib/snapd/apparmor/profiles/snap.k9s.k9s:
# Description: Can access specific personal files or directories in the
# users's home directory.
# This is restricted because it gives file access to arbitrary locations.
owner "@{HOME}/.kube{,/,/**}" rk,
I then verified the policy allows it with:
$ mkdir -p ~/.kube
$ echo "it worked!" > ~/.kube/foo
bash in the snap$ snap run --shell k9s
bash in the snap$ cat "$(getent passwd $USER | cut -d : -f 6)/.kube/foo"
it worked!
$ exit
This seems to be a problem with the snap. Note that the personal-files interface is granting access to the user’s actual home directory on the system. Perhaps the snap is trying to access $HOME which evaluates to ~/snap/k9s/<revision> and ~/snap/k9s/<revision>.kube doesn’t exist? If that is the case, see the getent command above to find the user’s home directory on the system.
For $HOME/.k9s, I actually suspect something similar where there is confusion in the snap about where $HOME/.k9s is (ie ~/.k9s vs ~/snap/k9s/<revision>.k9s. The snap is currently only allowed to use ~/snap/k9s/<revision>.k9s.
@jdstrand Thank you so much for this great analysis!! So there lies the problem. In both K9s and Popeye I am using a standard GO stdlib call to locate the user home dir and the tmp dir. I am guessing Kubernetes does the same to locate the .kube dir. Looking at the GO std lib call to get the user home dir it dives into the current process id struct to extract user info and picks out the user home from the c struct. I’ve cruised over the the kubectl snap and it looks like they are using classic mode probably for this very reason. What would be the process to request classic mode for both K9s and Popeye? I could work thru custom code to assess the user actual home dir while deployed as a snap, but that would prove brittle and less than ideal. Does this make sense or did I miss it? Thank you!!
@jdstrand Any update on this? I think the confinement regarding the $HOME is too strict and not what these tools require. K9s leverages the actual $HOME dir to store artifacts for the tool. Based on your previous comment this sandboxed home will change for every release ie ~/snap/k9s/rev.k9s and hence all K9s artifacts would be lost as the home dir is a moving target and will change on every new updates. Also K9s allows the user to shell out to kubectl which is found by checking user’s $PATH once again using GO std lib calls. This also turns out to be a dud given K9s current confinement. So what can I do to get this story wrapped up and have the K9s snap working correctly? This is slowing my tools adoption due to this currently inadequate confinement. Thank you!!
The snap is allowed to write to ~/snap/k9s/<revision>/* (including .kube) by default and the personal-files interface grants additional access to ~/.kube. If you can’t get your application to prefer one over the other via configuration or code change, simply create a symlink from ~/snap/k9s/<revision>/.kube to ~/.kube. You can create a small wrapper script or adjust your program to do something like:
snap_kube="$HOME/.kube"
myid=$(id -u)
myid_home=$(getent passwd $myid | cut -d : -f 6)
if [ ! -e "$snap_home ]; then
ln -sf "$myid_home" "$snap_home"
fi
In this manner, whether your application uses getpwent or $HOME, it should work.
@jdstrand Thank you for your input! Still seems less than ideal as the home is now a moving target on each release drop as the symlink will need to change. You did not answer my question regarding classic confinement. My understanding is all these problems will go away if k9s and popeye where to use it. Is this correct? And if so how can I proceed with the request?
Thank you!
This is handled by snapd. The script snippet I gave itself isn’t hardcoding any revisions so you should be ok there too. Also keep in mind that there is a current symlink down in ~/snap/k9s/current if you need it for some reason.
Since there are no restrictions put on your snap with classic confinement, you will not have any access problems. However, use of classic confinement is restricted and usually not needed. Access to dot files like we’ve been discussing is not a reason for classic confinement since either the snap can be adapted to work within the system or personal-files can be used (which is also restricted). This topic is discussing the suitability of certain uses of personal-files.
For your previous questions, SNAP_USER_DATA is versioned by snapd copies forward the data. If that doesn’t meet your use case, SNAP_USER_COMMON is not versioned and you can always choose to use that to store your artifacts. As for kubectl, you would need to ship that as part of your snap (which you would want to do anyway since there is no guarantee this will be available on the host even if you were using classic).
@jdstrand. Thank you for your reply and comments! I am still very confused. Most K9s user will most likely have kubectl either installed as snap or from the releases on their systems and most likely in their $PATH. K9s shells out to kubectl for some commands. I have installed kubectl via a snap. The user has a PATH env variable set that correctly contains /snap/bin. On the terminal which kubectl yields installed and available. This snap uses classic confinement and no extra configuration is necessary as it does find the .kubeconfig in the user’s home dir… Yet, If I run K9s and try to shell out to run a kubectl command I get kubectl is not in the PATH? So why is this failing? Sorry to be obtus here but it seems I am missing an important piece of information or I am completely missing it. I am now able to launch K9s by specifying where to find the kubeconfig but now anything I am trying to persist or launch is a total dud. Thank you for your response!
As mentioned, for your snap to work with strict confinement, you must ship your own kubectl for your snap to use. In strict confinement, your snap is not allowed to call kubectl from the host system (since it wouldn’t be available in your snap’s runtime environment) or from other snaps (commands in /snap/bin are not available for other snaps to use due to limitations of the security sandbox (particularly around seccomp). Instead, snaps that want to expose executables to other snaps must do so via the content interface).
You are correct that a classic confinement snap would not have this limitation, however, preferring to not ship a binary in your snap is not typically a reason for classic confinement.
There seems to be a disconnect in this conversation, so I’ll try to be clear. Classic confinement provides device ownership to the snap and we limit what is allowed to use classic confinement in the public store. Classic confinement is typically only granted for snaps in the public store when the requirements of the snap are understood to not fit within snapd’s current implementation. Thus far, the requirements of k9s as they pertain to classic confinement are that it needs access to $HOME/.kube and must use kubectl. Classic confinement is therefore not warranted since we’ve granted personal-files for the former and you can ship kubectl yourself (if you prefer to use the kubectl snap, you could approach the publisher of that snap to expose a content interface that your snap could use instead).
Hopefully this clarifies things a bit.
@jdstrand Thank you so much for the clarifications on this and for taking the time to respond! So I have a few issues with this which I think warrants for a classic confinement.
- Taking a kubectl dependency with the k9s snap means I have now to maintain a different release cycle to ensure an up to date deployment as kubectl revs.
- This also means that the user might choose to run a different version of kubectl on their system and yet be confined to the one the ships with K9s while in K9s. This doesn’t sit right with my expectations.
- K9s currently needs access to /tmp, .k9s, .kube and $PATH. Kubectl is one of the execs but we also launch a user preferred editor which per your explanation will mean, K9s needs to ship with these snaps too? An impossible requirement.
- I understand the link of the k9s snap home to current, but all dots files dependency must remain stable while the K9s tool revs. Hence linking to the `current release directory would require additional symlinks for the .dotfiles.
- Additionally the tmp dir is buried under the snap home and not as advertised as /tmp by the CLI. So that does mean that I would need to rewrite parts of the code whether a snap release is at play or not. Which is one of the thing I was trying to avoid in the first place.
So do I have enough here to make the case to request a classic confinement. I love Snapcraft and would love my users to experience K9s/Popeye in their best light without have to muck too much with configuration and settings ie snap install and go. I hope these make sense to your team and that your guys feel there is enough here to warrant a classic confinement.
Thank you for your time, patience and consideration!
@jdstrand Any updates on this? Getting lots of push back from my users not being able to use the snap install. Thank you!
If I’m not mistaken this can be done with xdg-open with xdg-desktop-portals from inside your snap if you plug the desktop interface. I just tested this and it works on Ubuntu, I’m unsure of the status of xdg-desktop-portals support in non-Ubuntu distros.
I don’t quite follow what you’re saying here, are you saying that the files inside the dot files will be binary incompatible between versions?
What do you need to access from /tmp that must be shared with the host’s version of /tmp?