How to delegate package creation in snapcraft.yaml to electron-builder?

snapcraft
1

I’m using the following snapcraft.yaml config file to build a snap package for my app in a confined environment:

name: boxy-svg
version: "3.73.0"
summary: Scalable Vector Graphics (SVG) editor.
description: "Boxy SVG project goal is to create the best tool for editing SVG files. For beginners as well as for professional web designers and web developers. On any device and operating system."
base: core18
grade: stable
confinement: strict
architectures:
  - build-on: amd64
    run-on: amd64
  - build-on: arm64
    run-on: arm64
  - build-on: armhf
    run-on: armhf
parts:
  electron-app:
    plugin: nil
    source: app/
    override-build: |
      npm install
      npm --prefix ./src install
      ./node_modules/.bin/electron-builder
      mv ./dist/*.snap $SNAPCRAFT_PROJECT_DIR
    build-snaps:
      - node/14/stable
    build-packages:
      - build-essential
      - git
      - libfontconfig-dev

The entire task of snap package creation is delegated to electron-builder tool which produces boxy-svg-amd64.snap package. However, after running snapcraft command there is also a second dummy boxy-svg_3.73.0_amd64.snap package (produced by snapcraft itself rather than by electron-builder). Is there some way to get rid of that dummy package? I suspect it’s going to cause problems when doing automated builds in the cloud.

2

I managed to prevent snapcraft from creating the dummy snap package by adding exit 1 at the end of override-build. This is an ugly hack though.

It looks like the dummy package is created in the last snap step, is there something like override-snap that would allow me to prevent the default behavior? Alternatively, can I configure Snapcraft Cloud Build (https://snapcraft.io/build) to not run the last snap step?

3

Using Electron Build like this is an antipattern, it’s designed to be run independently of Snapcraft, in the host environment, where it would make a snap itself, including for foreign CPU architectures.

However I do often recommend people use Snapcraft anyway, because it makes it easier to do certain changes, for example, you can more easily swap to Core20 and use the desktop extensions with a raw snapcraft.yaml file, than you could with Electron Build last time I checked. This solves some niche but troublesome problems, such as the odd issues people have with font rendering not working in certain setups.

But certainly you’ll need to make some changes, because the Snapcraft Build System (as in, the runners provided in the cloud to do the CI for you) will more than likely just abort and not work if it recieves the exit 1 command. The build system would have no reason to believe a valid snap was generated at all in this instance, because it didn’t complete the usual lifecycle.

Not working with Electron directly, I know that it’s possible to make Electron Build generate a folder that’s simply unpacked. I’m not sure whether you can select that as a target by itself, or whether it’s a byproduct of selecting AppImages as a target. But I’d take a look around in your build folder because it might already exist. If you find a folder (e.g, dist/linux-unpacked), it actually contains everything that’s needed, you just need to put it into the right place where snapcraft expects it, so you’d add to your override something like

mkdir $SNAPCRAFT_PART_INSTALL/myApp -p
mv dist/linux-unpacked $SNAPCRAFT_PART_INSTALL/myApp -r

You’d need to actually set up your app: stanza to make it work, along with any plugs, and probably the Gnome extensions, but this should let Snapcraft do it’s work from start to finish, letting you make proper use of the build environment while offering more control over the end result than Electron Build raw would.

And because you’re aiming for 3 architectures, it’s probably worth saying the name of the linux-unpacked folder varies per architecture too, so personally I actually use mv *unpacked just to catch them anyway.

1 Like
4

@James-Carroll Thanks for providing detailed instructions. I’m afraid all this is outside of my area of expertise (I’m a web developer). From my read of the other threads authoring a snapcraft.yaml file for an Electron app is non-trivial, especially when building packages for multiple architectures in the cloud which imposes additional restrictions. Most examples I found on GitHub are either outdated or are using an existing .deb package as a starting point.

What would be the best place to find a person to whom I could subcontract this work? Would you be interested in working on this yourself? Your task would be only to write the snapcraft.yaml file, I will provide a GitHub repo with all other assets and I will configure electron-builder to generate linux-unpacked directory.

5

Honestly, if you’ve made it this far I’d assume you could make it the full-stretch! I can see there’s already a lot of advanced understanding of snapcraft in what you’ve written above and it just looks like a case of being unable to apply it properly through lack of examples. Well luckily, I’ve a good example!

This is an Electron project I make a snap for myself, and hopefully it’ll make a good example for you. To help explain the directory layout,

snapcraft.yaml contains all your usual metadata, including the apps: stanza needed to make the snap actually runnable. plugins/joplin.py contains the equivilent of the override-build you used above, except by defining it as a custom plugin you can make the snapcraft.yaml slightly easier to understand by hiding some of the complexity elsewhere. And finally, gui/ contains the desktop icon and desktop metadata that actually puts your snap on the desktop so users can click on it.

If you take a look into the joplin.py plugin, adopting this for your own work, you’d remove the _apply_patches() from the build commands, because you aren’t using them. You’d then also customise the _build_commands() to work for your particular snap, but from what I see, it isn’t actually that far off from what you’ve already done.

On the snapcraft.yaml side, you would probably want to:

  • Change the name, description, etc.
  • Remove adopt-info and replace it with version: git
  • Remove the override-pull
  • Remove the entire xdg-open part
  • Keep the cleanup part (it makes the snap smaller!), but if you change the name of your source, ensure the after: variable is kept up to date
  • Ensure the apps: section reflects your specific application, the app should probably be named the same as your package name, and the command would need to be pointed at your own binary in the snap.
  • You could probably remove some of the plugs for your snap ( password-manager-service, avahi, cups, etc)
  • Add back in the architectures from your original snapcraft.yaml, so the build service knows which ones you’re actually interested in
  • Rename the plugin so it makes more sense (just change the filename and the value of plugin:

So while this might look more complex than what you already have (purely by lines of code), I’d be hopeful that you could base yourself on this example just fine. As long as Electron Builder is actually producing the linux-unpacked folder, this approach would work on multiple architectures and with the Snapcraft Build Service.

6

@James-Carroll I finally managed to make it work as per your hints (at least on my local machine, I haven’t tested cloud build yet).

I’m attaching below the config files in case someone else runs into similar problem (Electron apps usually have very similar structure). File open/save dialogs were not opening with core20, so I used core18 instead.

Directory structure:

  • snap/
    • snapcraft.yaml
    • gui/
      • boxy-svg.desktop
      • boxy-svg.svg (the app icon)
  • app.zip
    • package.json
    • src/

snapcraft.yaml:

name: boxy-svg
version: "3.73.1"
summary: Scalable Vector Graphics (SVG) editor.
description: |
  Boxy SVG project goal is to create the best tool for editing SVG files.
  For beginners as well as for professional web designers and web developers.
  On any device and operating system.
license: Proprietary
base: core18
grade: stable
confinement: strict
compression: lzo
architectures:
  - build-on: amd64
    run-on: amd64
  - build-on: arm64
    run-on: arm64
  - build-on: armhf
    run-on: armhf
parts:
  electron-app:
    plugin: dump
    source: app.zip
    source-type: zip
    override-build: |
      # Bugfix: https://forum.snapcraft.io/t/problems-with-remote-build/24373/2
      if [ -n "$http_proxy" ]; then
        export ELECTRON_GET_USE_PROXY=1
        export GLOBAL_AGENT_HTTP_PROXY="${http_proxy}"
        export GLOBAL_AGENT_HTTPS_PROXY="${http_proxy}"
      fi

      # Bugfix: Need to move the unarchived files manually because `source-subdir: app` does not seem to work
      mv ./app/* .

      # Install electron-builder and other dependencies from NPM
      npm install
      npm --prefix ./src install

      # Generate linux-unpacked directory containing Electron binaries
      ./node_modules/.bin/electron-builder

      # Copy over the linux-unpacked directory contents
      mkdir -p $SNAPCRAFT_PART_INSTALL/opt/boxy-svg || true
      cp -ar ./dist/linux-unpacked/* $SNAPCRAFT_PART_INSTALL/opt/boxy-svg/
    build-snaps:
      - node/14/stable
    build-packages:
      - build-essential
      - git
      - libfontconfig-dev
    stage-packages:
      - libappindicator3-1
      - libxss1
      - libnspr4
      - libnss3
      - libglu1-mesa
      - libnotify4
      - libnotify-bin
      - libpulse0
      - libsecret-1-0
  cleanup:
    plugin: nil
    build-snaps:
      - core18
      - gnome-3-28-1804
    override-prime: |
      # Running this code will reduce the final snap package size from ~180 MB to ~115 MB
      set -eux
      for snap in "core18" "gnome-3-28-1804"; do
        cd "/snap/$snap/current" && find . -type f,l -exec rm -f "$SNAPCRAFT_PRIME/{}" \;
      done
    after: [electron-app]
apps:
  boxy-svg:
    command: opt/boxy-svg/boxy-svg --no-sandbox
    plugs:
      - desktop
      - desktop-legacy
      - home
      - removable-media
      - x11
      - unity7
      - wayland
      - browser-support
      - network
      - opengl
      - audio-playback
    extensions: [gnome-3-28]
    environment:
      DISABLE_WAYLAND: 1
      TMPDIR: ${XDG_RUNTIME_DIR}
      PATH: ${SNAP}/usr/sbin:${SNAP}/usr/bin:${SNAP}/sbin:${SNAP}/bin:${PATH}
      SNAP_DESKTOP_RUNTIME: ${SNAP}/gnome-platform

boxy-svg.desktop

[Desktop Entry]
Name=Boxy SVG
Comment=Scalable Vector Graphics (SVG) editor
Exec=boxy-svg %u
Icon=${SNAP}/meta/gui/boxy-svg.svg
Type=Application
Terminal=false
Categories=Graphics
StartupWMClass=Boxy SVG
MimeType=image/svg+xml;image/png;image/jpeg;image/gif;image/webp;application/pdf;application/illustrator;
Keywords=SVG;Vector;Graphics;Editor;

package.json

{
  "name": "boxy-svg",
  "version": "3.73.1",
  "description": "Scalable Vector Graphics (SVG) editor.",
  "license": "Proprietary",
  "homepage": "https://boxy-svg.com/",
  "build": {
    "productName": "Boxy SVG",
    "appId": "com.boxy-svg.BoxySVG",
    "copyright": "© 2012-2021 Jarosław Foksa",
    "compression": "normal",
    "asar": true,
    "asarUnpack": [
      "**/*.node"
    ],
    "directories": {
      "app": "src",
      "output": "dist"
    },
    "linux": {
      "target": [
        "dir"
      ],
      "executableName": "boxy-svg"
    }
  },
  "devDependencies": {
    "electron": "16.0.5",
    "electron-builder": "22.14.5"
  }
}
7

Ahh, that’s actually a recent problem in the Electron environment. The explaination for it is there’s some bad interactions with their recent additions of XDG Desktop Portals in Electron 14+, depending on the versions of GTK being used. Gnome 3-38 (at least as packaged in the extension) does trigger it.

https://github.com/electron/electron/issues/31152

The workaround, that has to be applied at the application level rather than packaging level, is to make use of the asynchronous file open API’s which don’t suffer from the same bug.

So this isn’t strictly a snap specific issue and might be worth fixing if you have users in other environments where you can’t precisely control the GTK versions they’re using. If you do swap all the API’s to asynchronous mode, you’d probably find they work fine in Core20 with gtk 3.38

However aside from that specific error, it’s nice to see that you managed to find success with getting everything working from Snapcraft alone! Having looked at your snippets above, I can’t see anything I’d recommend changing (except maybe reconsidering Core20 if you work around the bug above, but Core18 has a lot of support still left in it, so it’s not essential by any means, and perhaps removing libsecret, since looking at your plugs I don’t think you’re using it). So congratulations, and welcome to the Snapcraft community :slight_smile:

8

(…) you can more easily swap to Core20 and use the desktop extensions with a raw snapcraft.yaml file (…). This solves some niche but troublesome problems, such as the odd issues people have with font rendering not working in certain setups.

Is this bug one of these odd issues that would be fixed by switching from core18 to core20? I received many complains from users of the existing core18-based app that open/save dialogs are showing gibberish text, but I was never able to reproduce it myself.

The workaround, that has to be applied at the application level rather than packaging level, is to make use of the asynchronous file open API’s which don’t suffer from the same bug.

I managed to make the file open/save dialogs work with core20 by switching to async Electron APIs as per your suggestion, but I also had to remove the “cleanup” part.

With the “cleanup” part present, the snap package size would be reduced from 207 MB to 134 MB, but the file open/save dialogs would fail to open with the following message printed in the terminal:

Gtk-WARNING **: 12:19:55.654: Can't open portal file chooser: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.portal.Desktop was not provided by any .service files

(…) perhaps removing libsecret, since looking at your plugs I don’t think you’re using it

electron-builder adds libsecret-1-0 by default, but you are right - it doesn’t seem to be required in my case so I removed it.

9

Yes, that’s the kind of issue that would be improved in Core20 with newer extensions. The issue is a bit complicated, but the fix comes down to a mixture of newer fontconfig libraries and improvements in the extensions to make their own private font caches and avoid using the hosts. I personally used to have a few reports a month about the font issues when the Joplin snap was using Core18 and the older extensions, since updating it to Core20 and the Gnome-3-38 extension when it was released, I haven’t had a single complaint about fonts in nearly a year.

It’s weird this shows up, unfortunately I’m not knowledgable enough to be able to say exactly why the cleanup part causes it. It might just happen to resolve itself on Core20 because I haven’t seen it myself. The cleanup part isn’t essential though, it does reduce the size of the snap and while this is obviously preferable, actually working properly is important too.

10

I’m attaching below the final configuration for my Electron-based app. It uses electron-builder and snapcraft to create core20-based snap packages for amd64, arm64 and armhf architectures. The packages are working fine and so far I haven’t recieved any complains from users.

Native node modules and building from GitHub repositories using Snapcraft’s remote build system are also supported with this configuration.

Thanks again @James-Carroll for all the hints, without your help and encouragement I would probably give up and drop the Linux support.

Directory structure

* snap/
  * snapcraft.yaml
  * gui/
    * boxy-svg.desktop
    * boxy-svg.svg (the app icon)
* app.zip
  * package.json
  * src/ (JavaScript code of the Electron app goes here)
    * main.js
    * package.json

snapcraft.yaml

name: boxy-svg
version: "3.73.1"
summary: Scalable Vector Graphics (SVG) editor.
description: |
  Boxy SVG project goal is to create the best tool for editing SVG files.
  For beginners as well as for professional web designers and web developers.
  On any device and operating system.
license: Proprietary
base: core20
grade: stable
confinement: strict
compression: lzo
architectures:
  - build-on: amd64
    run-on: amd64
  - build-on: arm64
    run-on: arm64
  - build-on: armhf
    run-on: armhf
parts:
  electron-app:
    plugin: dump
    source: app.zip
    source-type: zip
    override-build: |
      # Bugfix: https://forum.snapcraft.io/t/problems-with-remote-build/24373/2
      if [ -z ${http_proxy+x} ];
      then
        echo "Not using proxy"
      else
        echo "Using proxy"
        export ELECTRON_GET_USE_PROXY=1
        export GLOBAL_AGENT_HTTP_PROXY="${http_proxy}"
        export GLOBAL_AGENT_HTTPS_PROXY="${http_proxy}"
        npm config set proxy $http_proxy
        npm config set https-proxy $https_proxy
      fi
      # Bugfix: Need to move the unarchived files manually because `source-subdir: app` does not seem to work
      mv ./app/* .
      # Install electron-builder and other dependencies from NPM
      npm install
      npm --prefix ./src install
      # Generate linux-unpacked directory containing Electron binaries
      if [ ${SNAP_ARCH} = "armhf" ];
      then
        ./node_modules/.bin/electron-builder --armv7l
      else
        ./node_modules/.bin/electron-builder
      fi
      # Copy over the linux-unpacked directory contents
      mkdir -p $SNAPCRAFT_PART_INSTALL/opt/boxy-svg || true
      cp -ar ./dist/*-unpacked/* $SNAPCRAFT_PART_INSTALL/opt/boxy-svg/
    build-snaps:
      - node/14/stable
    build-packages:
      - build-essential
      - git
      - libfontconfig-dev
    stage-packages:
      - libappindicator3-1
      - libxss1
      - libnspr4
      - libnss3
      - libglu1-mesa
      - libnotify4
      - libnotify-bin
      - libpulse0
apps:
  boxy-svg:
    command: opt/boxy-svg/boxy-svg --no-sandbox
    plugs:
      - desktop
      - desktop-legacy
      - home
      - removable-media
      - x11
      - unity7
      - wayland
      - browser-support
      - network
      - opengl
      - audio-playback
    extensions: [gnome-3-38]
    environment:
      DISABLE_WAYLAND: 1
      GTK_USE_PORTAL: 1
      TMPDIR: ${XDG_RUNTIME_DIR}
      PATH: ${SNAP}/usr/sbin:${SNAP}/usr/bin:${SNAP}/sbin:${SNAP}/bin:${PATH}
      SNAP_DESKTOP_RUNTIME: ${SNAP}/gnome-platform

gui/boxy-svg.desktop

[Desktop Entry]
Name=Boxy SVG
Comment=Scalable Vector Graphics (SVG) editor
Exec=boxy-svg %u
Icon=${SNAP}/meta/gui/boxy-svg.svg
Type=Application
Terminal=false
Categories=Graphics
StartupWMClass=Boxy SVG
MimeType=image/svg+xml;image/png;image/jpeg;image/gif;image/webp;application/pdf;application/illustrator;
Keywords=SVG;Vector;Graphics;Editor;

app.zip/package.json

{
  "name": "boxy-svg",
  "version": "3.73.1",
  "description": "Scalable Vector Graphics (SVG) editor.",
  "license": "Proprietary",
  "author": "Jarosław Foksa",
  "homepage": "https://boxy-svg.com/",
  "build": {
    "productName": "Boxy SVG",
    "appId": "com.boxy-svg.BoxySVG",
    "copyright": "© 2012-2022 Jarosław Foksa",
    "compression": "normal",
    "asar": true,
    "asarUnpack": [
      "**/*.node"
    ],
    "directories": {
      "app": "src",
      "output": "dist"
    },
    "linux": {
      "target": [
        "dir"
      ],
      "executableName": "boxy-svg"
    }
  },
  "devDependencies": {
    "electron": "16.0.5",
    "electron-builder": "22.14.5"
  }
}

app.zip/src/package.json

{
  "name": "boxy-svg",
  "productName": "Boxy SVG",
  "version": "3.73.1",
  "description": "Scalable Vector Graphics (SVG) editor.",
  "license": "Proprietary",
  "author": "Jarosław Foksa",
  "homepage": "https://boxy-svg.com/",
  "main": "main.js",
  "dependencies": {
    "local-fonts-manager": "git+https://github.com/jarek-foksa/local-fonts-manager.git"
  }
}
1 Like