Why is my arm64 build pushed to the amd64 release slot?

Hi there.

I am building an Electron app using “electron-builder”, which can build snaps for amd64, arm64, and armhf architectures. But once I push those builds to the snap store the arm64 build is being pushed to the amd64 release slot.

According to “snapcraft -h” there is no way to manually decide where to push specific builds, that apparently is being decided by snapcraft.

That should not be the case right? Why does snapcraft think the arm64 build is equal to the amd64 one?

Best
ran

how does the architectures: block in your snapcraft.yaml look like ?

with “electron-builder” I don’t need to have a yaml, it does that job for me. I just need to run this script line in package.json and it will build the arm64 build:

And then the odd thing after uploading the build is this:

well, it seems to do that wrong then … (after all even electron-builder creates a meta/snap.yaml during build and it likely puts something wrong in there (most likely the host architecture on which the package was built))

From my quick experimentation here, Electron-Builder only produces valid ARM64 snaps if it uses destructive mode. If using the LXD and presumably Multipass backends (I can’t test multipass specifically due to lacking VM support in the test environment), it builds binaries matching the hosts architecture. (There is a warning thrown that --target-arch is being ignored, IMO this should be a hard error and abort the build, seeing as it’s clearly ignoring intentions, but regardless, the a snap file is produced none the less and is valid for the host architecture).

You can force destructive mode with export SNAPCRAFT_BUILD_ENVIRONMENT=host prior to electron-builder --linux snap --arm64, however it is highly encouraged that destructive mode is only used in CI environments where the build machine is constantly reset to a known good state, or else consistency in the resulting snap isn’t guaranteed. It’s also highly encouraged destructive mode is used on a machine where the host OS matches the base snap, which for Electron uses Core18 and thus Ubuntu 18.04.

Lacking actual ARM64 hardware, I couldn’t run the resulting Snap to see if it actually works so you might need to consider adding relevant stage libraries, but it should at least publish to the right architecture on the store.

1 Like

Note that this will build against AMD64 binaries if you’re building on an AMD64 host. I fail to understand how this is any different to building on an AMD64 multipass or lxd system (apart from also destroying your host ;-p).

It uses a precompiled Electron from upstream, the file command definitely reports it as ARM64 and the snap.yaml metadata is definitely ARM64. Whether it works or not I can’t say lacking something test the end result on, but there’s absolutely a substantial enough portion of ARM64 userland in there it should at least error out properly.

Edit: overall I expect this probably works for simple enough Electron apps that have minimal native dependencies (I.E, the runtime is all pure JS), but the second you have something complex the only valid solution might be to use an actual ARM64 system to build on.

1 Like

So I spent the night trying to figure out how the manual build process works using a yaml. And I have a huge headache now.

Anyway I created a yaml that does actually work at least locally on my machine. It creates a snap that I can start using snap run appname. And of course now I would like to use the automated build process that is triggered when I push the yaml to Github.

But when I do that and the snap being built remotely the same yaml provokes an error I just can’t get rid of.

Here is my yaml:

snapcraft.yaml
name: sleek
base: core18
version: '0.2.1'
summary: A simple todo manager based on the concept of todo.txt
description: |
  sleek is a simple todo manager based on the concept of todo.txt. That means you will use a simple but powerful syntax to add contexts and projects to your todos, you will prioritize them and set due dates. According to this input you will be able to filter and search your todos. There is no cloud integration, but as sleek writes your data to a plain text file, you can put it anywhere you want. That also means if you don't have sleek at hand or if you don't like it anymore you can just edit the todo.txt file with any text editor or other todo.txt application.

grade: stable
confinement: strict
icon: ./assets/icons/icon.png
license: MIT
architectures:
  - build-on: [amd64, arm64, s390x, ppc64el, armhf]
    run-on: [amd64, arm64, s390x, ppc64el, armhf]
apps:
  sleek:
    command: snap/command-chain/desktop-launch $SNAP/sleek/sleek --no-sandbox
    extensions: [gnome-3-28]
    plugs:
    - browser-support
parts:
  sleek:
    plugin: nodejs
    nodejs-package-manager: npm
    nodejs-version: 15.2.1
    build-snaps:
    - node/15/stable
    source: .
    override-build: |
      case $SNAPCRAFT_ARCH_TRIPLET in
        "i386-linux-gnu") ARCH="ia32";;
        "x86_64-linux-gnu") ARCH="x64";;
        "arm-linux-gnueabihf") ARCH="armv7l";;
        "aarch64-linux-gnu") ARCH="arm64";;
        *) echo "ERROR: electron does not support the '$SNAPCRAFT_ARCH_TRIPLET' architecture" && exit 1;;
      esac
        npm install electron-packager
        ./node_modules/.bin/electron-packager . --overwrite --platform=linux --arch=$ARCH --output=release-build --prune=true
        cp -v -R ./sleek-linux-$ARCH/ $SNAPCRAFT_PART_INSTALL/sleek
    stage-packages:
    - libnss3
    build-packages:
    - nodejs
    - npm
  cleanup:
    after:
      - sleek
    plugin: nil
    build-snaps:
      - core18
    override-prime: |
      set -eux
      for snap in "core18"; do  # List all content-snaps and base snaps you're using here
          cd "/snap/$snap/current" && find . -type f,l -exec rm -f "$SNAPCRAFT_PRIME/{}" \;
      done

This creates this error which interrupts the build process:

extract from build error
[27/Nov/2020:10:49:58 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 200 3935327 "-" "-"
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
[27/Nov/2020:10:50:02 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 200 44891837 "-" "-"
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
[27/Nov/2020:10:53:39 +0000] "CONNECT github.com:443 HTTP/1.1" 200 7206 "-" "node/v15.3.0 node-sass-installer/5.0.0"
[27/Nov/2020:10:53:47 +0000] "CONNECT github-production-release-asset-2e65be.s3.amazonaws.com:443 HTTP/1.1" 200 3404291 "https://github.com/sass/node-sass/releases/download/v5.0.0/linux-x64-88_binding.node" "node/v15.3.0 node-sass-installer/5.0.0"
npm ERR! code 1
npm ERR! path /build/sleek/parts/sleek/build/node_modules/electron
npm ERR! command failed
npm ERR! command sh -c node install.js

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-11-27T10_53_47_695Z-debug.log
Failed to run 'override-build': Exit code was 1.
Build failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/lpbuildd/target/build_snap.py", line 258, in run
    self.build()
  File "/usr/lib/python3/dist-packages/lpbuildd/target/build_snap.py", line 247, in build
    env=env)
  File "/usr/lib/python3/dist-packages/lpbuildd/target/build_snap.py", line 100, in run_build_command
    return self.backend.run(args, env=full_env, **kwargs)
  File "/usr/lib/python3/dist-packages/lpbuildd/target/lxd.py", line 537, in run
    subprocess.check_call(cmd, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['lxc', 'exec', 'lp-bionic-amd64', '--env', 'LANG=C.UTF-8', '--env', 'SHELL=/bin/sh', '--env', 'SNAPCRAFT_BUILD_INFO=1', '--env', 'SNAPCRAFT_IMAGE_INFO={"build-request-id": "lp-60699626", "build-request-timestamp": "2020-11-27T10:32:04Z", "build_url": "https://launchpad.net/~build.snapcraft.io/+snap/0bd466522b32144f4aca7878faa1937b/+build/1196432"}', '--env', 'SNAPCRAFT_BUILD_ENVIRONMENT=host', '--env', 'http_proxy=http://10.10.10.1:8222/', '--env', 'https_proxy=http://10.10.10.1:8222/', '--env', 'GIT_PROXY_COMMAND=/usr/local/bin/snap-git-proxy', '--', '/bin/sh', '-c', 'cd /build/sleek && linux64 snapcraft']' returned non-zero exit status 2.
Revoking proxy token...
RUN: /usr/share/launchpad-buildd/bin/in-target scan-for-processes --backend=lxd --series=bionic --arch=amd64 SNAPBUILD-1196432
Scanning for processes to kill in build SNAPBUILD-1196432

My package.json looks like this

package.json
{
  "name": "sleek",
  "productName": "sleek",
  "version": "0.2.1",
  "description": "A simple todo manager based on the concept of todo.txt",
  "synopsis": "A simple todo manager based on the concept of todo.txt",
  "category": "ProjectManagement",
  "keywords": [
    "todo.txt",
    "todo",
    "productivity",
    "tasks"
  ],
  "author": "Robin Ahle <mail@robbfolio.de>",
  "copyright": "Copyright © 2020 ${author}",
  "license": "MIT",
  "repository": "https://github.com/ransome1/sleek/",
  "icon": "assets/icons/512x512.png",
  "main": "src/index.js",
  "buildResources": "src",
  "build": {
    "appId": "com.sleek.desktop",
    "linux": {
      "icon": "assets/icons/512x512.png",
      "category": "ProjectManagement",
      "target": [
        "deb",
        "pacman",
        "freebsd",
        "rpm",
        "AppImage"
      ]
    },
    "win": {
      "target": [
        "portable"
      ],
      "icon": "assets/icons/512x512.ico"
    },
    "mac": {
      "target": "dmg",
      "icon": "assets/icons/512x512.icns",
      "category": "public.app-category.productivity"
    }
  },
  "appImage": {
    "artifactName": "${productName}-${version}-${arch}.${ext}"
  },
  "rpm": {
    "artifactName": "${productName}-${version}-${arch}.${ext}"
  },
  "freebsd": {
    "artifactName": "${productName}-${version}-${arch}.${ext}"
  },
  "snap": {
    "confinement": "strict",
    "grade": "stable"
  },
  "scripts": {
    "pack": "electron-builder --dir",
    "dist": "electron-builder -l --publish never",
    "snap": "electron-builder -l snap --arm64 --publish always",
    "css-build": "node-sass src/scss/ --output src/css/",
    "css-watch": "npm run css-build -- --watch",
    "start": "npm run css-watch & electron .",
    "build": "electron-packager ."
  },
  "dependencies": {
    "@fortawesome/fontawesome-free": "^5.15.1",
    "electron-squirrel-startup": "^1.0.0",
    "electron-util": "^0.14.2",
    "i18next": "^19.8.4",
    "i18next-browser-languagedetector": "^6.0.1",
    "i18next-fs-backend": "^1.0.7",
    "md5": "^2.3.0"
  },
  "devDependencies": {
    "bulma": "^0.9.1",
    "electron": "11.0.3",
    "electron-builder": "^22.9.1",
    "node-sass": "^5.0.0"
  }
}

Can anybody take a quick look at this and give me a little push into the right direction?

Best

you are mixing the nodejs plugin with a nodejs stage-snap, that will definitely cause conflicts … take a look how i do it here:

also … change your architectures to only contain “build-on” (similar to the above, list all arches you want to support)

1 Like

Thanks for the hint! My yaml is adjusted and builds successfully offline, unfortunately Snapcraft let’s me wait at the moment: Building soon :wink:

I’m excited.

@ogra, it worked :smiley:

And also it is building for all archs and pushing the builds to the right release slots. What a hassle, but finally I got it working. How fancy!

1 Like

@ogra please let me ask you another thing. My app needs access to the users file system to read or write a txt file. As the app users will access that txt file from many different applications (it will most likely even be dropped into a dropbox or syncthing folder) so it would be inconvenient for the user if the file is located in the snap folder.

I also understand I need to add an interface to achieve this. Is it correct I need to apply for this interface to get those privileges granted? Like post a request here? https://forum.snapcraft.io/c/store-requests

you can just write to the users home dir with the home interface plug added to the app, removable-media will allow you to write to /mnt and /media (if the user has write permissinos) …

beyond this there are portals you could use, but i havent seen any electron app using them yet so i cant really tell how hard/easy it is to implement support for them.

1 Like

The home interface plug worked perfectly well, thank you so much!

1 Like

So again the remote build process is crashing whereas the local process with the same yaml is working fine. I just can’t figure out why, it’s getting really frustrating. I just can’t see anything useful in the Snapcraft build report.

But the build report refers to a log that is probably be found in the remote remote VM. And as I don’t want to bother your guys all the time, please let me ask you if there is way to access this log and figure out what is going wrong.

This is the current log.

Build log on Snapcraft
> [28/Nov/2020:09:31:52 +0000] "CONNECT registry.npmjs.org:443 HTTP/1.1" 200 44891864 "-" "-"
> npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
> [28/Nov/2020:09:31:58 +0000] "CONNECT github.com:443 HTTP/1.1" 200 7212 "-" "node/v15.3.0 node-sass-installer/5.0.0"
> [28/Nov/2020:09:32:07 +0000] "CONNECT github-production-release-asset-2e65be.s3.amazonaws.com:443 HTTP/1.1" 200 3404291 "https://github.com/sass/node-sass/releases/download/v5.0.0/linux-x64-88_binding.node" "node/v15.3.0 node-sass-installer/5.0.0"
> [28/Nov/2020:09:32:08 +0000] "CONNECT github.com:443 HTTP/1.1" 200 7227 "-" "-"
> npm notice 
> npm notice New patch version of npm available! 7.0.14 -> 7.0.15
> npm notice Changelog: <https://github.com/npm/cli/releases/tag/v7.0.15>
> npm notice Run `npm install -g npm@7.0.15` to update!
> npm notice 
> [28/Nov/2020:09:32:55 +0000] "CONNECT github-production-release-asset-2e65be.s3.amazonaws.com:443 HTTP/1.1" 200 76611972 "-" "-"
> npm ERR! code 1
> npm ERR! path /build/sleek/parts/sleek/build/node_modules/electron
> npm ERR! command failed
> npm ERR! command sh -c node install.js
> 
> npm ERR! A complete log of this run can be found in:
> npm ERR!     /root/.npm/_logs/2020-11-28T09_32_55_672Z-debug.log
> Failed to run 'override-build': Exit code was 1.
> Build failed
> Traceback (most recent call last):
>   File "/usr/lib/python3/dist-packages/lpbuildd/target/build_snap.py", line 257, in run
>     self.pull()
>   File "/usr/lib/python3/dist-packages/lpbuildd/target/build_snap.py", line 223, in pull
>     env=env)
>   File "/usr/lib/python3/dist-packages/lpbuildd/target/build_snap.py", line 100, in run_build_command
>     return self.backend.run(args, env=full_env, **kwargs)
>   File "/usr/lib/python3/dist-packages/lpbuildd/target/lxd.py", line 537, in run
>     subprocess.check_call(cmd, **kwargs)
>   File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
>     raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['lxc', 'exec', 'lp-bionic-amd64', '--env', 'LANG=C.UTF-8', '--env', 'SHELL=/bin/sh', '--env', 'SNAPCRAFT_LOCAL_SOURCES=1', '--env', 'SNAPCRAFT_SETUP_CORE=1', '--env', 'SNAPCRAFT_BUILD_INFO=1', '--env', 'SNAPCRAFT_IMAGE_INFO={"build-request-id": "lp-60719828", "build-request-timestamp": "2020-11-28T09:28:32Z", "build_url": "https://launchpad.net/~build.snapcraft.io/+snap/0bd466522b32144f4aca7878faa1937b/+build/1197571"}', '--env', 'SNAPCRAFT_BUILD_ENVIRONMENT=host', '--env', 'http_proxy=http://10.10.10.1:8222/', '--env', 'https_proxy=http://10.10.10.1:8222/', '--env', 'GIT_PROXY_COMMAND=/usr/local/bin/snap-git-proxy', '--', '/bin/sh', '-c', 'cd /build/sleek && linux64 snapcraft pull']' returned non-zero exit status 2.
> Revoking proxy token...
> RUN: /usr/share/launchpad-buildd/bin/in-target scan-for-processes --backend=lxd --series=bionic --arch=amd64 SNAPBUILD-1197571
> Scanning for processes to kill in build SNAPBUILD-1197571
snapcraft.yaml

name: sleek
base: core18
version: ‘0.2.1’
summary: A simple todo manager based on the concept of todo.txt
description: |
sleek is a simple todo manager based on the concept of todo.txt. That means you will use a simple but powerful syntax to add contexts and projects to your todos, you will prioritize them and set due dates. According to this input you will be able to filter and search your todos. There is no cloud integration, but as sleek writes your data to a plain text file, you can put it anywhere you want. That also means if you don’t have sleek at hand or if you don’t like it anymore you can just edit the todo.txt file with any text editor or other todo.txt application.

grade: stable
confinement: strict
icon: ./assets/icons/icon.png
license: MIT
compression: lzo
architectures:

  • build-on: [amd64, armhf, arm64, s390x, ppc64el]
    apps:
    sleek:
    command: snap/command-chain/desktop-launch $SNAP/sleek/sleek --no-sandbox
    environment:
    DISABLE_WAYLAND: 1
    TMPDIR: $XDG_RUNTIME_DIR
    PATH: “$SNAP/usr/sbin:$SNAP/usr/bin:$SNAP/sbin:$SNAP/bin:$PATH”
    SNAP_DESKTOP_RUNTIME: $SNAP/gnome-platform
    extensions: [gnome-3-28]
    plugs:
    • home
      parts:
      sleek:
      plugin: nil
      build-environment:
    • PATH: “$PATH:$SNAPCRAFT_PART_BUILD/node_modules/.bin”
      build-snaps:
    • node/15/stable
      source: .
      override-build: |
      if [ -n “$http_proxy” ]; then
      export ELECTRON_GET_USE_PROXY=1
      export GLOBAL_AGENT_HTTP_PROXY="${http_proxy}"
      export GLOBAL_AGENT_HTTPS_PROXY="${http_proxy}"
      fi
      npm -g install
      electron-packager . --overwrite --platform=linux --output=release-build
      –prune=true --out $SNAPCRAFT_PART_INSTALL/
      mv $SNAPCRAFT_PART_INSTALL/$SNAPCRAFT_PROJECT_NAME-linux-*
      $SNAPCRAFT_PART_INSTALL/sleek
      stage-packages:
    • libnss3
      cleanup:
      after: [ sleek ]
      plugin: nil
      build-snaps: [ gnome-3-28-1804 ]
      override-prime: |
      set -eux
      cd /snap/gnome-3-28-1804/current
      find . -type f,l -exec rm -f $SNAPCRAFT_PRIME/{} ;
      for CRUFT in bug lintian man; do
      rm -rf $SNAPCRAFT_PRIME/usr/share/$CRUFT
      done
      find $SNAPCRAFT_PRIME/usr/share/doc/ -type f -not -name ‘copyright’ -delete
      find $SNAPCRAFT_PRIME/usr/share -type d -empty -delete

looks a bit like it tries to download an unnamed package ("-" “-”)

do you perhaps have a typo in your package.json or some such ?

also why did you add -g to the npm install call (that might write modules in the wrong place)

Now it’s working again and I am trying to compare my settings with the ones I got before. The only difference I see is that I moved the “electron-packager”: “^15.1.0” entry in my package.json from “dependencies” to “devDependencies”. That will most likely have been the problem.

1 Like

And now it’s failing again, no adjustments on the yaml or package.json. Snapcraft is playing with me, this cannot be true. I give up and will build locally again :wink:

@ogra: The unnamed packages ("-" “-”) also occurred in the successfully released snaps. So I don’t think thats the problem.

Anyway, thank you a lot for your support.