/usr/bin/fold got permission denied but /usr/bin/fmt is working fine. What's happened?


#1

What plug allows a snap to access the binaries made available in the OS, e.g.: /usr/bin? I have been digging through:

  1. Interfaces
  2. https://docs.snapcraft.io/reference/interfaces

and still can’t find any available solution. Am I missing something out? :face_with_raised_eyebrow:


I’m facing a weird issue with my recent snap upload where the snap is not able to access /usr/bin/fold, which is a part of Busybox commands on amd64 Ubuntu bionic.

However, I can access rm, echo etc nicely. The issue log is available here:
https://gitlab.com/ZORALab/BaSHELL/issues/16


UPDATE
Earlier Question: is there a plug to access /usr/bin?
Answered: you shouldn’t … - by Lin-Buo-Ren


#2

The answer is probably “you shouldn’t”, the resources you should access are in the Ubuntu Core, your snap, and the very limited set of things exposed from the host system.

/usr/bin/fold is provided by the coreutils package, and should be provided by the core snap at /snap/core/current/usr/bin/fold.


#3

Thanks for the reply.

Is there a way to access /snap/core/current/usr/bin/fold without needing to modify the source code for the snap?

From the source codes perspective, this is a Snapcraft specific problem and the deb packages from launchpad has no issue.

Currently, what I can think of is using environment variables to hold fold path and add:

fold=$(which fold)
if [[ "$SNAP" != "" ]]; then
   fold=/snap/core/current/usr/bin/fold
fi

#4

I’m not sure how it is implemented but it seems that you can’t directly point the script to access files under /snap/core/current/*.

You can verify it by run snap run --shell _snap_name_ and try to access these files.

I suspect that this is probably a snapcore/snapd’s bug, which doesn’t expose /snap/core/current/usr/bin/fold…, maybe it can be workarounded by adding the coreutils package to stage-packages ?


#5

If we let you run it from core, then core needs to always ship it. So each one is added on a case-by-case basis.

Also: inside the (non-classic) snap, /usr/bin is /snap/core/current/usr/bin, so stop doing what you’re trying to do with the path.


#6

I extracted the snap environment variables using the env command.

XDG_VTNR=7
LC_PAPER=en_DK.UTF-8
XDG_SESSION_ID=c2
LC_ADDRESS=en_DK.UTF-8
XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/holloway
CLUTTER_IM_MODULE=xim
LC_MONETARY=en_DK.UTF-8
GLADE_PIXMAP_PATH=:
GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
TERM=xterm-256color
XDG_MENU_PREFIX=xfce-
SNAP_USER_COMMON=/home/holloway/snap/bashell/common
SHELL=/bin/bash
VTE_VERSION=5202
SNAP_CONTEXT=<hide as I suspect it is sensitive data>
TMPDIR=/tmp
DERBY_HOME=/usr/lib/jvm/java-8-oracle/db
GOBIN=/home/holloway/bin
LC_NUMERIC=en_DK.UTF-8
WINDOWID=37748774
SNAP_REEXEC=
USER=holloway
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
CLUTTER_BACKEND=x11
QT_ACCESSIBILITY=1
LC_TELEPHONE=en_DK.UTF-8
GLADE_MODULE_PATH=:
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
KICAD_PATH=/usr/share/kicad
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
SNAP_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void
SNAP_COMMON=/var/snap/bashell/common
DEFAULTS_PATH=/usr/share/gconf/xubuntu.default.path
SESSION_MANAGER=local/holloway-spectre:@/tmp/.ICE-unix/1733,unix/holloway-spectre:/tmp/.ICE-unix/1733
SNAP_USER_DATA=/home/holloway/snap/bashell/x1
XDG_CONFIG_DIRS=/etc/xdg/xdg-xubuntu:/etc/xdg:/etc/xdg
SNAP_DATA=/var/snap/bashell/x1
DESKTOP_SESSION=xubuntu
KIGITHUB=https://github.com/KiCad
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
TEMPDIR=/tmp
QT_QPA_PLATFORMTHEME=gtk2
QT_IM_MODULE=fcitx
LC_IDENTIFICATION=en_DK.UTF-8
PWD=/home/holloway/src/gitlab.com/zoralab/bashell
XDG_SESSION_TYPE=x11
XMODIFIERS=@im=fcitx
JAVA_HOME=/usr/lib/jvm/java-8-oracle
LANG=en_US.UTF-8
MANDATORY_PATH=/usr/share/gconf/xubuntu.mandatory.path
GDM_LANG=en_US
S_COLORS=auto
LC_MEASUREMENT=en_DK.UTF-8
PAPERSIZE=a4
GDMSESSION=xubuntu
SNAP_REVISION=x1
HOME=/home/holloway/snap/bashell/x1
SHLVL=2
XDG_SEAT=seat0
LANGUAGE=en_US
SNAP_NAME=bashell
SNAP_COOKIE=<HIDE AS I SUSPECT IT IS SENSITIVE DATA>
GTK_OVERLAY_SCROLLING=0
BUNDLE_PATH=/home/holloway/.gem
LOGNAME=holloway
XDG_SESSION_DESKTOP=xubuntu
QT4_IM_MODULE=fcitx
XDG_DATA_DIRS=/usr/share/xubuntu:/usr/share/xfce4:/usr/local/share:/usr/share:/var/lib/snapd/desktop:/var/lib/snapd/desktop:/usr/share
J2SDKDIR=/usr/lib/jvm/java-8-oracle
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
SNAP_ARCH=amd64
LESSOPEN=| /usr/bin/lesspipe %s
GOPATH=/home/holloway
SNAP_VERSION=1.3.0
DISPLAY=:0.0
XDG_RUNTIME_DIR=/run/user/1000/snap.bashell
GLADE_CATALOG_PATH=:
XDG_CURRENT_DESKTOP=XFCE
SNAP=/snap/bashell/x1
J2REDIR=/usr/lib/jvm/java-8-oracle/jre
GTK_IM_MODULE=fcitx
LESSCLOSE=/usr/bin/lesspipe %s %s
LC_TIME=en_DK.UTF-8
XAUTHORITY=/home/holloway/.Xauthority
COLORTERM=truecolor
LC_NAME=en_DK.UTF-8
_=/usr/bin/env

I tested the coreutils packages (and it requested libc6 package). After installing the built snap, the bashell unables to run and yield: Segmentation fault (core dumped).

Also, during build, it has a lot of warnings, like:

warning: working around a Linux kernel bug by creating a hole of 2097152 bytes in ‘/tmp/tmp5j99l32i’
warning: working around a Linux kernel bug by creating a hole of 2097152 bytes in ‘/tmp/tmphh95zh6b’
warning: working around a Linux kernel bug by creating a hole of 2097152 bytes in ‘/tmp/tmpi_aoun50’
warning: working around a Linux kernel bug by creating a hole of 2097152 bytes in ‘/tmp/tmp4t49upz6’
warning: working around a Linux kernel bug by creating a hole of 2097152 bytes in ‘/tmp/tmp2e8uiyal’

@chipaca, what is the proper way to do? Based on the shell inside the snap, the fold is given:

$ which fold
/usr/bin/fold

$ which echo
/bin/echo

$ which ls
/bin/ls

I wouldn’t want to alter the original software since it has no such issue.


UPDATE
Appearently, fmt works, and the location it points is:

$ which fmt
/usr/bin/fmt

Both fmt and fold originated from the same coreutils package. Why fold is not working is unknown.


#7

the easiest thing would be to ship fold (just that binary, not the whole coreutils) in your snap.

I don’t understand what you’re doing. I don’t know much snapcraft, but I don’t think it ever pulls in libc6 unless you tell it to. It sounds like you’re building with, or shipping conflicting versions of libc6. What exactly are you doing?

I don’t know what you’re building when you see those errors, but if it’s Qt, you might be needing an upstream fix; in any case it’s not (directly) related to the tools you’re using to build.


#8

fmt is whitelisted, so you can run it from core. fold is not, so you need to ship your own copy (or request that we add it to the whitelist, and then wait for that updated whitelist to ship).


#9

Basically I add the stage-packages, with the snapcraft.yaml, to build fold (alongside with its siblings):

  stage-packages:
      - libc6
      - coreutils

and rebuild it. Then when I run my app after snap install, it went into segmentation fault.


I thought of that but I strongly believe it is a workaround, especially when developing mutli-architecture software.


I believe we arrived to the source of the problem now: fold is not whitelisted. Therefore snap fails. Also, I wasn’t aware there is such a thing as well and I assume all commands listed in the BusyBox are all white-listed.

Is there a website where we can check the white-list commands? This site should be very helpful for future shell scripting inside the snap.


#10

This is unnecessary, simply add a new part with stage-packages: coreutils and

    stage:
        - usr/bin/fold

should be enough.


#11

For a list of what’s “whitelisted” (and maybe I should point out that that was a simplification on my part),
a good place to start would be


#12

I tried. As far as I understand, you’ll need to access the fold by modifying the path to $SNAP/usr/bin/fold, per this thread: Exposing folders of files without specifying each of them individually or expose the fold as a command (which is not desired).

Unless I’m missing something where I can set /usr/bin/fold into an internal program for internal consumption. Is there a way?

The whitelist / blacklist of the commands affecting Busybox list is completely out-of-the-box and not documented. That’s the actual cause of the issues.


That’s a good help! If there is any similar scripting project that needs BusyBox commands, I’ll create a request.


#13

there should be no need to as long as you use snapcrafts “apps” entry … this auto-generates a wrapper with a PATH entry pointing to $SNAP/bin, $SNAP/usr/bin etc etc … so “fold” should be executed from $SNAP (this indeed gets hairy if your binary usees full paths (which it shouldnt if you want to write portable code)).

in case you want more complex setups you can also take a look at the experimental layouts feature …

i.e. something like:

passthrough:
  layout:
    /usr/bin/fold
      bind: $SNAP/usr/bin/fold

should bind your shipped “fold” binary to the /usr/bin path and allow you to run it.


Call for testing - cheat
#14

Simply stage the executable is enough as in the runtime PATH="$SNAP/usr/sbin:$SNAP/usr/bin:$SNAP/sbin:$SNAP/bin:$PATH"


#15

Check out $SNAP/command-command_name.wrapper for the assignment.


#16

FYI, I’ll add fold in the the next batch of updates, but in the meantime, you need to ship it in your snap.


#17

Yeap. If we use the apps entry, it will expose the fold as an app externally. In some situation, we might not want it. No worries, I can still live with $SNAP variables for time being.


Thanks for sharing :wink:!

passthrough is definately new and came to realize, I’m on the bleeding edge. Reading through:

  1. Support for passthrough into snap.yaml
  2. https://snapdocs.labix.org/snapcraft-yaml-reference/4276

That’s definitely a way but it will create extra documentation step and exhibits enough complexity for packaging process. These 2 are enough to say a packager had failed its manifesto. By ideal user experience, we should always restrict the amount of user action to 1 and only:

$ snap install <software>

The rest are packager’s job.


Thank you! Am already editing 1.3.1 release to use fmt. No hush.


#18

One thing I think you’ve misunderstood: the commands in core are not busybox, they’re the full GNU (in the case of coreutils) toolset, from 16.04 in the case of core (18.04 in core18).


#19

My bad. I didn’t clarify properly.

I use BusyBox command list as my checklist mainly because on some single-computer-board (arm/x86), the OS (e.g. Yocto) usually deploys BusyBox, which has limited commands. You’re right too! Those commands are from GNU coreutils on our computers (x86 amd64, i386).

To ensure bash script is working across systems, I try to maintain the bash script to use commands within the BusyBox (left alone bash has the POSIX Compliant vs. sh to deal with too).

For software that works on all available arch, yet compliance: Go binaries :fist_right::fist_left: Snapcraft == :rainbow:


Action taken: retitled again to match the context for next person search.