Hello there! For the past few days, I have been working on snapping Apptainer. Apptainer is the most popular container platform/system for high-performance computing clusters. I have a working snap built, however, to distribute via the store, I need quite a few things for the snap need to be approved. What needs to be approved is broken into the following sections below:
Justification for classic confinement
First, Apptainer needs to be granted permission to be released with classic confinement. This is because the fuse-support interface does not fully support what Apptainer needs. Apptainer needs to be able to mount FUSE file systems outside of SNAP_USER_{DATA,COMMON} and SNAP_{DATA,COMMON}. Apptainer also needs the ability to bind mount directories depending on the end-users use case. Also, Apptainer may need to perform mounts or mount images on or from shared file systems such as GPFS, Lustre, or NFS.
Justification for suid binaries
Second, the Apptainer snap needs to be allowed to ship suid binaries. The currently needed suid binaries are the following:
-
newuidmap - Needed so that Apptainer can build container images in fakeroot mode. Apptainer uses newuidmap to create its fakeroot sandbox so that non-root users do not need sudo privileges to build images.
-
newgidmap - Needed so that Apptainer can build container images in fakeroot mode. Apptainer uses newuidmap to create its fakeroot sandbox so that non-root users do not need sudo privileges to build images.
-
squashfuse - Needed so that Apptainer can mount squashfs archives as FUSE file systems. Apptainer uses squashfs under the hood to create its container images, and then squashfuse is used so that non-root users can mount images without needing sudo privileges.
Justification for aliases
Lastly, the Apptainer snap needs two aliases granted:
-
singularity - This alias is needed because the singularity symbolic link is used to provide backwards compatibility to older scripts and programs that still use the original singularity executable to interact with SIF (Singularity Image Format), .img, squashfs files. Apptainer is an open-source fork of the original Singularity container platform that has since been split into SingularityCE and SingularityPro.
-
run-singularity - Alias to apptainer run [run options…] <container>. This executable is used to provide backwards compatibility to older scripts and programs that use the run-singularity executable to run the %runscript bundled within the Apptainer image. runscripts are commonly used to perform tasks such as launch Jupyter Server, launch IDEs, run compiled simulations, operate on datasets, etc.
Conclusion
Please let me know if there is any further information required, and I look forward towards hearing your response. If code auditing is required, you may find my snapcraft.yaml file here: https://github.com/canonical/apptainer/blob/main/snap/snapcraft.yaml
As per the Process for reviewing classic confinement snaps, can you please outline which of the supported categories for classic confinement that apptainer fits within?
Regarding suid binaries, this is a very rare thing to grant since it requires a manual override within the review-tools package which is used to validate snaps on upload to the store - would it be possible to just suggest to users that they need to use apptainer with sudo instead? That way it is clear to them which commands require root privileges from apptainer?
Regarding the aliases, singularity conflicts with the singularity game within the Ubuntu archive but I don’t think these two applications are likely to be installed together on many machines so this is unlikely to cause a conflict.
+1 from me for the aliases singularity and run-singulariry for apptainer.
Hey Alex! Thank you for getting back to me! Let me answer your questions:
Classic confinement
Not sure if I am wording this correctly, but essentially snapd does not quite support what I need Apptainer to do. Apptainer needs to be able to mount FUSE file systems outside of SNAP_USER_{DATA,COMMON} and SNAP_{DATA,COMMON}; the fuse-control interface provided by snapd does not allow Apptainer to mount FUSE file systems outside of those directories.
Also, Apptainer is used to run workloads on HPC systems where traditional user’s do not exist. i.e. users may not exist on compute node as they may on the head node, and the user’s home directory may not exist under /home. Their home directory may be elsewhere such as under /data, /mnt, or some other arbitrary location. The user will also have a different home setup such as having scratch and work directories mounted at locations outside of home and then symlinked into the home directory. TL;DR this is because user files and directories are propagated over networked file systems such as NFS, GPFS, Lustre, etc. Please let me know if you need further explanation or if you have any suggestions.
Why I need suid binaries
While directing users to just use sudo when interacting with the snap works, it breaks the UX of Apptainer. Apptainer is meant to used by users who do not have sudo privileges. HPC cluster users are almost never, if ever, given access to sudo. If the Apptainer snap were to be distributed where it could only be used with sudo, it would be barely used.
To provide some context, I did not want to have to use suid binaries (I understand the security concerns + seen other forum posts regarding the same issues), but it was the only way to get newuidmap, newgidmap, and squashfuse to work. Without these suid binaries, Apptainer will be unable to build images with fakeroot or mount containers.
I could try modifying the capabilities of these executables rather than applying a suid bit, but I am unsure if that just creates the same problem. Another solution could be that I direct sysadmins to install uidmap and squashfuse with apt, but then I would require classic so that the Apptainer snap can access the provided executables outside of its runtime.
Hey @alexmurray. Just wanted to ask if there are any updates on this request for Apptainer. Thanks!
Regarding classic confinement, the requirements are understood, and it would seem that Apptainer fits within the “HPC or orchestration agents/software for running workloads on systems without traditional users where the systems are otherwise managed outside of the agent” existing category for classic confinement.
Regarding the second question of suid binaries, if the snap is then granted classic confinement and it ships suid binaries I really feel this is a bad combination as in general there is nothing now confining this application that then has user->root escalation privileges OOTB. So I wonder, if the only real issue regarding strict confinement is the mount location for FUSE filesystems, then could this either be documented for your users so they are aware of this limitation, then we could look at granting suid override but keeping the snap strictly confined.
Or we grant classic confinement but not the suid overrides.
Which use-case do you feel would be better to support?
Hey again Alex. Thank you for getting back to me!
To be honest, I would prefer to be granted the suid overrides rather than classic confinement. FUSE mounting and fakeroot are critical features for non-root Apptainer users to have access to. I will investigate how I can strictly confine Apptainer.
I will report back once I have made some progress on strict confinement for Apptainer.
1 Like
Hey @nuccitheboss,
Then I am removing this request from our classic confinement review queue.
When you have updates, please create a new forum topic so we can properly discuss the new accesses required.
Hey there!
Yes, sounds good. I will reopen a new thread when I finish making the necessary changes to the snap. Thank you!
1 Like
