I have a weird one and I haven’t been able to fully grok what’s going on so I’m hoping someone here might have a suggestion.
In the node snap, we have yarn
being bundled. Yarn follows basically the same process as npm for installing Node.js packages. Packages can have “lifecycle scripts” to aid in the installation process, the most common of these being a post-installation script that either compiles native addon code (mostly using the node-gyp
command) or downloading a pre-compiled binary of the addon (node-pre-gyp
or prebuild
being two common tools for this). Lifecycle scripts are all executed using sh -c
, I’m pretty sure npm still does this and Yarn certainly does.
npm works just fine from within its snap, installing and compiling and downloading and everything you expect.
Yarn, on the other hand, doesn’t like some very common lifecycle scripts. It has a fancy console “reporter” that prints information about install progress as it’s happen, usually obscuring stdout from child processes involved in executing these lifecycle scripts, showing it line-by-line and writing over it with status messages when done. This isn’t normally a problem and I can’t find anything about it being a problem on any platform for Yarn.
I can’t get any useful error information other than the process exiting with a non-zero exit status. If I try and handle stdio to get output by hacking Yarn I get a successful build. A true heisenbug. My fix currently is to hack Yarn with sed
with the ‘install’ phase of the snap to force replacement of the “reporter” with “inherited” child process stdio (which just means that stdio, stderr and stdin are piped to/from the parent). When I do that it all goes fine, but the nice reporter output is messed up with child process output interrupting it. So it’s not ideal.
So, my current guess is that there’s something about apparmor, or other containment mechanism of snaps that is treating stdout differently to a standard non-contained application. Yarn doesn’t handle child process streams in the same way that npm does and I could believe that child process stdout buffers when it should be discarded.
To reproduce, here’s a modified version of the node snapcraft.yaml that uses an upstream binary rather than recompiling (so you don’t have to wait for a compile). The Yarn fix is commented out on the last two lines. So build this, install it, add an alias for yarn
and try yarn add buffertools
(uses node-gyp
, an automatic part of Yarn & npm when there’s a binding.gyp), and yarn add sqlite3
(uses node-pre-gyp
to download/or compile the addon) and yarn add leveldown
(uses prebuild
to essentially do the same thing as node-pre-gyp
).
name: node
version: '8.10.0'
summary: Node.js
description: |
A JavaScript runtime built on Chrome's V8 JavaScript engine. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient. Node.js' package ecosystem, npm, is the largest ecosystem of open source libraries in the world. https://nodejs.org/
grade: stable
confinement: classic
apps:
node:
command: bin/node
npm:
command: bin/npm
yarn:
command: bin/yarn.js
parts:
node:
#plugin: make
plugin: dump
source-type: tar
#source: https://nodejs.org/download/release/v8.10.0/node-v8.10.0.tar.gz
source: https://nodejs.org/download/release/v8.10.0/node-v8.10.0-linux-x64.tar.gz
#build-packages:
# - g++
# - make
# - python2.7
#prepare: |
# ./configure --prefix=/ --release-urlbase=https://nodejs.org/download/release/ --tag=
stage: [usr/*,bin/*,lib/*]
install: |
mkdir -p $SNAPCRAFT_PART_INSTALL/etc
echo "prefix = /usr/local" >> $SNAPCRAFT_PART_INSTALL/etc/npmrc
yarn:
source-type: tar
source: https://yarnpkg.com/latest.tar.gz
plugin: dump
# Yarn has a problem with lifecycle scripts when used inside snap, they don't complete properly, with exit code !=0.
# Replacing the spinner with proper stdio appears to fix it.
#install: |
# sed -i "s/var stdio = spinner ? undefined : 'inherit';/var stdio = 'inherit';/" $SNAPCRAFT_PART_INSTALL/lib/cli.js
There’s one more weird thing about this … I can get node-gyp
builds to work without my Yarn hack if I hack the way that node-gyp
is called by Yarn inside the snap. To do this, I need to stage [python2.7,python] in the package and add the following to the install
for the ‘node’ part:
echo -n '#!/bin/sh\nexport PYTHON=$SNAP/usr/bin/python\necho $PATH\nexec "$SNAP/bin/node" "$SNAP/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "$@"' > $SNAPCRAFT_PART_INSTALL/bin/node-gyp
chmod 755 $SNAPCRAFT_PART_INSTALL/bin/node-gyp
ln -sf ../../../../../bin/node-gyp $SNAPCRAFT_PART_INSTALL/lib/node_modules/npm/bin/node-gyp-bin/node-gyp
What this essentially does is replace the default node-gyp
launch script that Yarn reaches for with one that is slightly more direct. But all it’s doing is replacing:
#!/usr/bin/env sh
if [ "x$npm_config_node_gyp" = "x" ]; then
node "`dirname "$0"`/../../node_modules/node-gyp/bin/node-gyp.js" "$@"
else
"$npm_config_node_gyp" "$@"
fi
(that first branch is the most common)
But this only fixes node-gyp, not node-pre-gyp, prebuild and anything else non-standard. Plus, I don’t understand why this might work, at all. Perhaps it’s to do with using /usr/bin/env to find sh
(this is a common pattern in Node-land, particularly for finding Node: #!/usr/bin/env node), replacing it with a more direct call and also the direct PYTHON
reference.