I have a snap running on node.js but I ran into build issues at runtime as I’m missing environment variables related to python (such as $PYTHON) and make (such as $CPLUS_INCLUDE_PATH).
Python and make are both installed, but their env variables are not set. Is there a way to set these rather than clutter the yaml with and environment: section?
Looks like it boils down to not being able to access “/usr/lib/x86_64-linux-gnu/libmvec_nonshared.a”. This file exists in the /snap/mySnap/x1/usr/lib/x86_64-linux-gnu but not in the /usr/lib/x86_64-linux-gnu:
Are you actually trying to compile and link stuff at runtime? I would have expected your snap to come without a compiler and instead bundle the binary objects, ready to execute.
Yes it is
And all nessecary bits are in the snap folder, but make (or maybe node-gyp) is trying to fetch the files from /user/lib in root where not all files are accessable (unless in classic)
I realize that is unconvetional, but I still need to compile the files at runtime, I’d have thought using LD_LIBRARY_PATH would have solved it, but regardless of what I do (or try to do), it still tries to get the libmvec_nonshared.a file from /user/lib
LD_LIBRARY_PATH is for the dynamic linker. Why do you think you need to do this at runtime? Dragging the whole linker, compiler, headers and development libraries into this is going to be messy.
The snap in an IoT application running micro services that may change over time. These services are normally just JavaScript’s (npm packages) where we don’t have issues, but in some cases they are bundled with C/C++ files and needs compiling and this is where we run into problems
dynamically building bits of your snap is not a typical use case so you will need to add a lot of manual work …
… to get the environment vars right (i.e. have LD_LIBRARY_PATH point to $SNAP/usr/lib ) you will likely have to either create app entries for each of your binaries used to compile the npm modules (this automatically creates wrappers for them using the right LD_LIBRARY_PATH), or if you do not want to expose them to the outside world (assuming you dont want a <snapname>.gcc ), move all the binaries from <binary> to <binary>.real and create wrappers called <binary> that sets the variables and in the end executes the .real file…
this will be a hell lot of work i suppose…
additionally you will have to make sure that your builds happen in $SNAP_DATA (or any other writable place the snap has) and that your node binaries have the target install dir (which has to be writeable too) added to their search path …
If there’s a way to deploy your app and then have it snapped then do that. Building this at runtime is really more complex and is IMO against the spirit of the snap package where everyone gets the same bits.
Thanks for a great reply, and yes all "dynamic part of the app is written to $SNAP_DATA. My .yaml includes stage- and build packages (python2.7, build-essential) along with node packages (node-gyp@latest). They are all installed, except for necessary env paths.
As the issues seems to relate to gcc/make, is there a special package I should/could add?
But whatever those are you can compile all of them ahead of time and just enable a subset right? In the end node (or whatever else you are using) will dlopen a .so file that you need to compile ahead of time.
if there are dynamic bits needed snap wont be in your way.
while this is definitely massively complex, you can surely do it and it is not different to i.e. dynamically loading random app plugins for any desktop app …
so i wouldnt say it is “against the spirit” … quite the opposite, it shows that snap is definitely flexible enough to allow it if you put enough work in …
you would have to add all packages your plugins require to build to “stage-packages”. then you would need to add a wrapper for each and every binary these packages include …
then patch the build system (probably even dynamically at runtime if the build scripts/Makefiles come along with the plugin) to:
use a writable dir of your snap to build in
install the resulting binaries in a writable place
and then make sure your binaries that make use of them know where to find them (by adjusting the search paths).
We get the same error: make: Entering directory '/root/snap/mysnap/x1/demo/build' SOLINK_MODULE(target) Release/obj.target/addon.node /snap/mysnap/x1/usr/bin/ld: cannot find /usr/lib/x86_64-linux-gnu/libmvec_nonshared.a
So we search the entire $SNAP structure for libmvec_nonshared.a and found it in $SNAP/usr/lib/x86_64-linux-gnu/libm.so
The content of the file looks like this: OUTPUT_FORMAT(elf64-x86-64) GROUP ( /lib/x86_64-linux-gnu/libm.so.6 AS_NEEDED ( /usr/lib/x86_64-linux-gnu/libmvec_nonshared.a /lib/x86_64-linux-gnu/libmvec.so.1 ) )
As the path is not relative the snap (although included in the $SNAP/usr/lib/x86_64-linux-gnu folder), the snap can not access the file (the file is there but the snap is not granted privileges).
Question: Is there a way to either affect this behavior in the snapcraft process (for instance through letting the snap think it’s living in the root), or ultimately give the snap full access to the /usr directory? Also, when you (@ogra) mention “wrapper”, what would that be?
well, wrappers to set a poper LD_LIBRAY_PATH around anything that might need to access libs … or in the quoted case above a hack to the makefile to prefix any /usr/lib path with $SNAP …
And all nessecary bits are in the snap folder, but make (or maybe node-gyp) is trying to fetch the files from /user/lib in root where not all files are accessable (unless in classic)