Strange discrepancy between root and non-root user when using snap package

snapcraft
1

I have been working on creating a snap package for the Apptainer container platform for HPC systems, and I have been noticing some odd discrepancies between when I test the package as root v.s. a non-root user. When I interact with the snap package as root (build containers, launching services, pulling images, etc.) everything works perfectly, but when I try to use the Apptainer snap as a non-root user, say ubuntu, everything starts to go awry.

For example, here is what happens when I try to build a container as user ubuntu:

ubuntu@apptainer-snap:~$ apptainer build http.sif http.def
FATAL:   newuidmap was not found in PATH (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin), required with fakeroot and unprivileged installation when user is in /etc/subuid

or when I try to execute a process inside the container:

ubuntu@apptainer-snap:~$ apptainer exec http.sif python3 -V
FATAL:   container creation failed: mount hook function failure: mount /proc/self/fd/3->/snap/apptainer/x1/var/apptainer/mnt/session/rootfs error: while mounting image /proc/self/fd/3: squashfuse exited with status 127: /snap/apptainer/x1/usr/bin/squashfuse: error while loading shared libraries: libsquashfuse.so.0: cannot open shared object file: No such file or directory

Oddly enough, both the library libsquashfuse.so.0 and executable newuidmap exist within the snap package under /snap/apptainer/current, and both of the commands above work perfectly however when I prepend them with sudo.

Would anyone happen to know what the issue might be here with the discrepancy between root v.s. non-root? Here is the in-progress snapcraft.yaml file. I am currently having the snap package install with classic confinement, but I hardly imagine that this is the cause of my issues:

name: apptainer
version: 1.1.3
license: BSD-3-Clause-LBNL
summary: An open-source container system for High-Performance Computing.
description: |
  Apptainer is the most widely used container system for High-Performance
  Computing. It is designed to execute applications at bare-metal performance
  while being secure, portable, and 100% reproducible. Apptainer is an
  open-source  project with a friendly community of developers and users.
  The user base continues to expand, with Apptainer now used across industry
  and academia in many areas of work.

base: core22
grade: devel
confinement: classic
compression: lzo

architectures:
  - build-on: amd64

parts:
  apptainer:
    plugin: dump
    source: https://github.com/apptainer/apptainer.git
    source-tag: v1.1.3
    build-packages:
      - build-essential
      - libseccomp-dev
      - pkg-config
      - uidmap
      - squashfs-tools
      - squashfuse
      - fuse2fs
      - fuse-overlayfs
      - fakeroot
      - cryptsetup
      - curl
      - wget
      - golang
    stage-packages:
      - build-essential
      - libseccomp-dev
      - pkg-config
      - uidmap
      - squashfs-tools
      - squashfuse
      - fuse2fs
      - fuse-overlayfs
      - fakeroot
      - cryptsetup
      - curl
      - wget
      - golang
    override-build: |
      craftctl default
      cd $CRAFT_PART_SRC
      ./mconfig --prefix=$CRAFT_PART_INSTALL
      make -C builddir
      make -C builddir install

  cleanup:
    after: [apptainer]
    plugin: nil
    override-prime: |
      set -eux
      cd $CRAFT_PRIME
      rm -rf \
        *.md builddir cmd dist docs e2e \
        examples go.mod go.sum internal \
        makeit mconfig mlocal pkg scripts tools

apps:
  apptainer:
    command: bin/apptainer
    environment:
      LD_LIBRARY_PATH: $SNAP/usr/lib/x86_64-linux-gnu/:$LD_LIBRARY_PATH
      PATH: $SNAP/usr/bin/:$PATH

  singularity:
    command: bin/singularity

  run-singularity:
    command: bin/run-singularity

Any words of wisdom are appreciated!

2

Solved my problem. Here is how I did it:

Not finding executables on the path: I needed to patch the application’s configuration file to include /snap/apptainer/current/bin and /snap/apptainer/current/usr/bin in its path lookup. The application has a module to looks up executables by searching $PATH, but it was not including the snap paths.

Not finding shared libraries: I needed to enable patchelf for core22. patchelf is not yet available in snapcraft’s stable channel, but it is available in the edge channel. After I switched snapcraft to the edge, I needed to add the following tags to my snapcraft.yaml file:

build-attributes:
  - enable-patchelf

Leaving this here in case anyone has the same problem in the future.

2 Likes