Multiple problems when build on Mac

snapcraft
1

I’m trying to build snap on my Mac and got multiple problems (it builds without issues on Linux).

I had this project structure:

.
├── buildroot
├── camera
├── controller
└── snap
    └── snapcraft.yaml

that builds buildrood based camera firmware as asset for nodejs based controller application i.e. on Linux I’m building without issue in ./snap/ but when tried to build on Mac it didn’t mount …/buildroot, …/camera and build failed. Solution was to move snapcraft.yaml in the project root like:

.
├── buildroot
├── camera
├── controller
└── snapcraft.yaml

And to change paths in snapcraft.yaml from:

camera-firmware:
    plugin: kbuild
    override-pull: |
        export SNAP_ROOT="../../../"
        make -C $SNAP_ROOT/../buildroot/ O=$SNAPCRAFT_PART_SRC BR2_EXTERNAL=../camera/ scanner_camera_defconfig

to:

camera-firmware:
    plugin: kbuild
    override-pull: |
        export SNAP_ROOT="../../../project"
        make -C $SNAP_ROOT/buildroot/ O=$SNAPCRAFT_PART_SRC BR2_EXTERNAL=../camera/ scanner_camera_defconfig

With these changes the buildroot camera firmware was successfully build but the node firmware failed with these errors:

sh: 1: node-gyp: Permission denied
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: netlink-notify@github:treedys/netlink-notify (node_modules/netlink-notify):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Error while executing:
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: /usr/bin/git ls-remote -h -t ssh://git@github.com/treedys/netlink-notify.git
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: 
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fatal: Could not change back to '/root/parts/treedys-controller/build': Permission denied
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: 
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: exited with error code: 128

First one is caused by lack of github credentials (which I have on my Mac):

snapcraft-treedys-controller ../parts/treedys-controller/build# /usr/bin/git ls-remote -h -t ssh://git@github.com/treedys/netlink-notify.git
The authenticity of host 'github.com (140.82.118.4)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,140.82.118.4' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Second one seems to be related to the fact the snapcraft uses root account for multipass builds and somehow npm messes with user file ownership. For example .npm directory is owned by multipass user which is highly suspicious:

snapcraft-treedys-controller # ls -al /root/
total 60
drwx------ 12 root      root      4096 Aug  8 07:17 .
drwxr-xr-x 23 root      root      4096 Aug  8 05:40 ..
-rw-r--r--  1 root      root       121 Aug  8 05:40 .bashrc
drwxr-xr-x 19 root      root      4096 Aug  8 06:45 .buildroot-ccache
drwxr-xr-x  3 root      root      4096 Aug  8 05:40 .cache
drwx------  4 root      root      4096 Aug  8 05:44 .config
drwxr-xr-x  6 multipass multipass 4096 Aug  8 07:17 .npm
-rw-r--r--  1 root      root       148 Aug 17  2015 .profile
drwx------  2 root      root      4096 Aug  8 07:32 .ssh
drwxr-xr-x  4 root      root      4096 Aug  8 05:41 parts
drwxr-xr-x  2 root      root      4096 Aug  8 05:41 prime
drwxr-xr-x  1 root      root       352 Aug  8 07:51 project
drwxr-xr-x  3 root      root      4096 Aug  8 05:40 snap
drwxr-xr-x  2 root      root      4096 Aug  8 05:41 stage
-rw-r--r--  1 root      root      1921 Aug  8 05:41 state

So my questions are:

  1. How to pass from snapcraft to multipass which directory to mount to avoid moving snapcraft.yaml from ./snap to ./ ?
  2. How to get SNAP_ROOT environment variable pointing to the directory where the snapcraft.yaml file is or where is the multipass mount i.e. the idea is to have stable path to base all relative part paths on.
  3. How to pass .ssh credentials from the host machine to the multipass VM (i.e. adding ssh -A parameter for example)?
  4. Do we need to build in multipass as root user? Why not to build as multipass or ubuntu or user defined user?
2

Hi @palavrov,

How are you running snapcraft in your scenario? It should work just fine from your project root (no need to move snapcraft.yaml anywhere).

You should not rely on “escaping” the build stages this way (I mean SNAP_ROOT). Each step gets built in its respective space and the after keyword is there for you to be able to have dependencies between the different parts.

Could you share your project whole?

As for git authentication, there’s work in Snapcraft happening to support this, but in the mean time you could put it in as a submodule and not a git type part, this way snapcraft will just copy the checked out tree from your host.

3

Damn … I was executing snapcraft from ./snap/ directory - didn’t know/expect that it could work from ./ when there is ./snap/snapcraft.yaml

Let’s me check how it will work like that and will report progress/issues - the build is really slow and will take a while.

It is a legacy project initially based entirely on buildroot - for lot of reasons we decide to migrate to snap - the buildet image file is used for network boot of RPi’s that are connected to the computer that runs that snap i.e. it is a runtime not compile time dependency so don’t need to use after dependencies.

About the “escaping” - didn’t find a better way to handle buildroot integration - it is a git submodule on same level as our image customization - and we were building several images like that i.e. was better to share the same buildroot source for all.

Let the build finish and will share the snapcraft.yaml with latest changes.

4

FYI snapcraft export SNAPCRAFT_PROJECT_DIR gives you the abspath to the root of the project (https://snapcraft.io/docs/t/parts-environment-variables/12271)

5

Finally everything seems to work now … took me forever to (snap)craft everything i.e. two weeks.

@sergiusens - thanks, SNAPCRAFT_PROJECT_DIR was exactly what I was looking for. But it was not available until I added base: core to snapcraft.yaml - took me few hours/days to discover that - yes, I know that there is a warning that it is missing but would like to have more explicit warning that I’m trying to use something that will not work until add base core.

Next problem was that the VM memory was not enough to build everything. And the build miserably failed with random errors not related to the lack of free memory at all. It would be good to monitor VM free memory and warn when it fails below 10%. Running snapcraft with SNAPCRAFT_BUILD_ENVIRONMENT_MEMORY=8G snapcraft solves the problem but I would like to put that directly into snapcraft.yaml because it is mandatory for that snap - something like:

build-environment:
    - memory: 8G
    - disk: 200G

Next problem was with nodejs plugin - current npm implementation doesn’t work well when it is running as root - sometimes it tries to impersonate as root parent user which causes confusing ownership errors. That was real pain to be debugged and fixed - took me more than week - solution was to set explicitly correct SUDO_UID, SUDO_GID, SUDO_USER to make root parent of root. Final working part is:

parts:
    treedys-controller:
        plugin: nodejs
        nodejs-version: '10.13.0'
        nodejs-package-manager: 'npm'
        source: controller/app              # For checking source updates only
        build-environment:
            - SUDO_UID: '0'
            - SUDO_GID: '0'
            - SUDO_USER: 'root'
        override-pull: |
            snapcraftctl pull               # call snapcraft to fetch node binaries and app source
            # cleaning package.json from `devDependencies` and `scripts`
            cat package.json | jq '{name, version, description, main, bin, author, license, private, dependencies, optionalDependencies}' > new.json
            mv new.json package.json
            # remove npm artefacts
            rm -rf .gitignore .npmignore .npmrc node_modules package package-lock.json
            # remove build artefacts
            rm -rf app.webpack.config.js src webpack.config.js www
        build-packages:
            - 'build-essential'
            - 'jq'
            - 'git'
            - 'python'

At the beginning I was trying to export SUDO_UID=0 but it was not working when npm is invoked by snapcraftctl pull - just because it executes from the context of snapcraft not from the override-pull - this is not obvious and needs to documented to avoid any confusion + adding example how to pass environment variables to snapcraftctl. I would like to be able to set different environment variables for each step (pull, build, stage, etc) that can be overridden. Another pain was that nodejs needs additional build-packages to work properly with native modules - worth to be added by default (build-essential, git, python, etc).

1 Like
6

For the former, you can use build-environment in the part to set specific environment variables for the given part where set.

And the latter, doing that would essentially burden individuals and/or projects using pure modules only. Wouldn’t these steps be needed on a vanilla install in any case?

7

Sorry for the late reply … too much other things to do … so:

Yes, as said I figured out how to solve the problem with build-environment - my point was that it was not obvious at all and giving example in the documentation would make the live of someone else with that problem much easier. This is very specific snapcraft nodejs plugin problem that happens when someone uses non-published in npm (or forked for some reason, private, etc) modules from github.

Assume that you mean the build-packages suggestion - build-essential, git, python are nearly always necessary when dealing with large nodejs based projects. It wouldn’t hurt to install them or at least mention in the documentation that if you are using native nodejs modules or non npm modules (note that sometimes you even couldn’t now that you are using such) you’ll need to have these build-packages. Usually these packages are already installed on the developer machine but they are not present in the minimalistic multipass VM.