Migrating between bases

A base snap is a special kind of snap that provides a run-time environment with a minimal set of libraries that are common to most applications. They’re transparent to users, but they need to be considered and specified when building a snap.

See Base snaps for details on how use and specify them.

Each base snap is built from a corresponding Ubuntu LTS release and migrating a snap from one base to the next gives the snap access to newer packages, extended support, and the latest Snapcraft features, including plugins and extensions.

The complexity of the migration process is directly linked to both dependencies in the snap’s snapcraft.yaml and the base snap versions being migrated between.

At its simplest, migrating from one base snap to another requires only that the base keyword is updated:

- base: core18
+ base: core20

But further changes will most likely be needed, and what these are will depend on the original base and the packages that are bundled alongside the application. The most common required changes are described below:


Updating from no or old bases

Migrating a snap from having no base, or base: core, to core18 or core20, for example, is a more involved process than going from core18 to core20.

This is because when building a snap with an old base, Snapcraft will operate in compatibility mode.

Compatibility mode is essentially a prior (2.43-era) version of Snapcraft, and will lose the functionality of newer releases. See Features incompatible with bases for details.

Package names

The build-packages and stage-packages sections in a snap’s snapcraft.yaml specify which packages need to be incorporated during the build and stage parts of the Parts lifecycle, and described in Build and staging dependencies.

When no base or core is specified, packages from the Ubuntu 16.04 LTS archive are used at build and stage time. The core18 base will use packages from the Ubuntu 18.04 LTS archive, whereas the core20 base will consume packages from the Ubuntu 20.04 LTS archive, and package names can change between releases.

Package name example: Irssi

    stage-packages:
-      - libperl5.22
+      - libperl5.26

In the above example, the name of the Perl library package changed due to a version bump. The best way to resolve these issues is to first build your snap on the destination base system, either via snapcraft or a virtual machine/LXD container, and update each unresolved package in turn with the new equivalents.

Architectures

The architectures keyword defines a set of both build and run architectures:

architectures:
  - build-on: amd64
    run-on: amd64

Snaps that produce i386 builds are supportable for the lifetime of Ubuntu 16.04 LTS or Ubuntu 18.04 LTS when using the core or core18 snaps as the base, but base: core20 does not support the i386 architecture.

Publishers who want to move to ‘base: core20’ must drop builds for the i386 architecture since it isn’t unavailable. Supported core20 architectures are listed below:

architectures:
  - build-on: amd64
  - build-on: arm64
  - build-on: armhf
  - build-on: ppc64el
  - build-on: s390x

For potential approaches to maintain an i386 build of a snap, see How best to handle i386 when moving to core20.

Environment variables

Environment variables are often used in snaps to ensure binaries are able to find loadable modules or libraries which reside inside the snap at runtime. Sometimes this results in path names which require updates due to directory name changes between versions.

Environment variables example: Irssi

    environment:
-        PERL5LIB:  "$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl-base/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl5/5.22/:$SNAP/usr/share/perl5/:$SNAP/usr/share/perl/5.22.1/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl/5.22/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl/5.22.1/"
+        PERL5LIB:  "$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl-base/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl5/5.26/:$SNAP/usr/share/perl5/:$SNAP/usr/share/perl/5.26.1/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl/5.26/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl/5.26.1/"

When a package name changes or is updated, it’s worth checking to make sure no environment variables are dependent on a path related to an older name, as with the above path.

Remote parts and Extensions

In some snaps remote parts may have been used to share configuration across multiple snaps and to reduce the local snapcraft.yaml complexity.

These parts are defined elsewhere, and would be incorporated at build time. This functionality is deprecated, so remote parts should be pasted directly into the snapcraft.yaml or referenced from their source repository.

Example of pasted remote part: Mr Rescue

 parts:
   mrrescue:
-    after:
-      - desktop-glib-only
+    desktop-glib-only:
+	   build-packages:
+	     - libglib2.0-dev
+	   plugin: make
+	   source: https://github.com/ubuntu/snapcraft-desktop-helpers.git
+	   source-subdir: glib-only
+	   stage-packages:
+	     - libglib2.0-bin

Alternatively for some desktop applications it may be appropriate to switch to using an extension, which simplifies the snapcraft.yaml further. This is covered in Snapcraft Extensions.

Example migration to an Extension: Xonotic

 parts:
   xonotic:
-    after:
-      - desktop-glib-only
 apps:
   xonotic:
-    command: desktop-launch $SNAP/Xonotic/xonotic-linux-sdl.sh
+    extensions: [gnome-3-34]
+    command: Xonotic/xonotic-linux-sdl.sh

In the above example, we remove the reference to a remote part desktop-glib-only and instead use the extensions section to use the gnome-3-34 extension, which replaces the functionality of the remote part.

Extension naming

Not all extensions work on all bases. For example, on core18 , use the gnome-3-34 extension and on core20 use gnome-3-38. See Supported extensions for further details.

Example showing core20-only Gnome extension: Dwarf Fortress

 parts:
   tarball:
-     after: [desktop-gtk3]
 apps:
   dwarffortress:
-    command: desktop-launch $SNAP/wrapper.sh
+    extensions: [gnome-3-38]
+    command: wrapper.sh

Audio interfaces

For applications which play or record audio, the interface names have changed.
Previously the pulseaudio interface was used for both playback and recording of audio. This has been replaced by audio-playback and audio-record:

Example audio interface update: Xonotic

 apps:
   xonotic:
     plugs:
-      pulseaudio
+      audio-playback     

Note that to ensure privacy, audio-playback is automatically connected but audio-record is not.

Application publishers who believe audio-record should be automatically connected on install (such as for an audio recording application) should start a thread in the store-requests category on the Snapcraft forum asking for it.

Version scripts

The top level version-script option has been deprecated in favour of adopt-info. This requires that you specify adopt-info with a reference to the part in which the version data (and some other metadata) may be set.

Within the parts section, use snapcraftctl set-version to define the snapcraft project version number used at build time.

Example replacing version-script with adopt-info: Cointop

-version-script: git -C parts/cointop/build rev-parse --short HEAD
+adopt-info: cointop
 parts:
   cointop:
+    override-pull: |
+      snapcraftctl pull
+      snapcraftctl set-version $(git rev-parse --short HEAD)     

See Using external metadata for further details.

Plugin name changes

The following plugin names have changed across Snapcraft releases:

nodejs / npm

The nodejs plugin is now npm.

e.g. wethr

 parts:
   wethr:
-    plugin: nodejs
+    plugin: npm

Plugin syntax

Plugin changes can be queried with the snapcraft help <plugin name> --base <base name> command:

$ snapcraft help npm --base core20
Displaying help for the 'npm' plugin for 'core20'.
[...]

You can also list plugins for a specific base with snapcraft list-plugins --base <base name>:

$ snapcraft list-plugins --base core20
Displaying plugins available for 'core20'
autotools  catkin  catkin-tools  cmake  colcon  dump  go  make
meson nil  npm  python  qmake  rust

The following plugins have changed their syntax across Snapcraft releases.

npm

The npm plugin uses npm-node-version instead of node-engine to specify the version of upstream npm to be used at build time.

Example npm plugin syntax change: wethr

 parts:
   wethr:
-    node-engine: "10.14.1"
+    npm-node-version: "10.14.1"

autotools

The Autotools plugin has migrated options from configflags to autotools-configure-parameters.

Example Autotools plugin syntax changes: Inadyn

 parts:
   libconfuse:
    plugin: autotools
-    configflags: ['--prefix=/usr', '--disable-examples', '--disable-static']
+    autotools-configure-parameters: ['--prefix=/usr', '--disable-examples', '--disable-static']

go

The go plugin no longer requires the go-importpath to be specified. A go-channel should be specified.

Example Go plugin syntax changes: slack-term

 parts:
   slack-term:
     plugin: go
-      go-importpath: github.com/erroneousboat/slack-term
+      go-channel: latest/stable

Application definitions

Paths

Snapcraft now requires explicit paths to be specified for binaries listed in the apps stanza:

Example update adding explicit paths: wethr

 apps:
   wethr:
-    command: wethr
+    command: bin/wethr

command-chain

Rather than specify command followed by a long list of space-separated executables, they can now be listed with the command-chain option:

Example of command being replaced by command-chain: Atom

 apps:
   atom:
-    command: bin/launcher ${SNAP}/usr/share/atom/atom
+    command-chain: 
+      - bin/launcher
+    command: usr/share/atom/atom

Examples summary

3 Likes

Here’s my notes from doing a bit of mirgation from no base specified or core used to core18 or core20. Feel free to integrate these into the documentation as you see fit. Happy to help flesh these out if needed.

Preamble

When migrating a snap from having no base specified, or having base: core to core18 and/or core20 there are behaviour and syntax changes in snapcraft which need to be taken care of. There’s other changes which are outlined below. This isn’t a comprehensive guide, but a set of tips for migrating snaps to core20.

Build and Stage Packages

The build-packages and stage-packages section specifies packages from the Ubuntu Archive to be incorporated during the build and stage parts of the snapcraft lifecycle.

With no base specified, or core used, packages from the Ubuntu 16.04 LTS archive will be used at build and stage time. The core18 base will use packages from the Ubuntu 18.04 LTS archive, whereas the core20 base will consume packages from the Ubuntu 20.04 LTS archive.

Package names can change between releases.

e.g. Irssi

    stage-packages:
-      - libperl5.22
+      - libperl5.26

Environment Variables

Environment variables are often used in snaps to ensure binaries are able to find loadable modules or libraries which reside inside the snap at runtime. Sometimes this results in path names which require updates due to directory name changes between versions.

e.g. Irssi

    environment:
-        PERL5LIB:  "$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl-base/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl5/5.22/:$SNAP/usr/share/perl5/:$SNAP/usr/share/perl/5.22.1/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl/5.22/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl/5.22.1/"
+        PERL5LIB:  "$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl-base/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl5/5.26/:$SNAP/usr/share/perl5/:$SNAP/usr/share/perl/5.26.1/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl/5.26/:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/perl/5.26.1/"

Remote Parts & Extensions

In some snaps ‘remote parts’ may have been leveraged to reduce the snapcraft.yaml complexity. These parts are defined elsewhere, and would be incorporated at build time. This functionality is deprecated, so remote parts should be pasted directly into the snapcraft.yaml or referenced from their source repository.

e.g. Mr Rescue

 parts:
   mrrescue:
-    after:
-      - desktop-glib-only
+    desktop-glib-only:
+	   build-packages:
+	     - libglib2.0-dev
+	   plugin: make
+	   source: https://github.com/ubuntu/snapcraft-desktop-helpers.git
+	   source-subdir: glib-only
+	   stage-packages:
+	     - libglib2.0-bin

Alternatively for some desktop applications it may be appropriate to switch to using an extension, which simplifies the snapcraft.yaml further.

e.g. Xonotic

We remove the reference to a remote part desktop-glib-only and instead use the extensions section to use the gnome-3-34 extension, which replaces the functionality of the remote part.

Note: On core18 (which I used here) it’s gnome-3-34 extension, and on core20 it’s gnome-3-38. Check which extensions are available with snapcraft list-extensions.

 parts:
   xonotic:
-    after:
-      - desktop-glib-only
 apps:
   xonotic:
-    command: desktop-launch $SNAP/Xonotic/xonotic-linux-sdl.sh
+    extensions: [gnome-3-34]
+    command: Xonotic/xonotic-linux-sdl.sh

e.g. Dwarf Fortress

 parts:
   tarball:
-     after: [desktop-gtk3]
 apps:
   dwarffortress:
-    command: desktop-launch $SNAP/wrapper.sh
+    extensions: [gnome-3-38]
+    command: wrapper.sh

Migrate Audio Interfaces

For applications which play or record audio, the interface names have changed. Previously the pulseaudio interface was used for both playback and recording of audio. This has been replaced by audio-playback and audio-record.

e.g. Xonotic

 apps:
   xonotic:
     plugs:
-      pulseaudio
+      audio-playback     

Note that audio-playback is automatically connected, audio-record is not. Application publishers who believe audio-record should be automatically connected on install (such as for an audio recording application) should start a thread in the “Store Requests” category on the snapcraft forum asking for it.

Version Scripts

The top level version-script option is deprecated in favour of adopt-info. Specify adopt-info referring to the part in which version (and some other metadata) may be set.

Within the parts section, use snapcraftctl set-version to define the snapcraft project version number used at build time.

e.g. Cointop

-version-script: git -C parts/cointop/build rev-parse --short HEAD
+adopt-info: cointop
 parts:
   cointop:
+    override-pull: |
+      snapcraftctl pull
+      snapcraftctl set-version $(git rev-parse --short HEAD)     

Plugin name changes

Plugin names may change in newer releases of snapcraft.

nodejs / npm

The nodejs plugin is now npm.

e.g. wethr

 parts:
   wethr:
-    plugin: nodejs
+    plugin: npm

Plugin Syntax Changes

From an empty directory, or in a project where base: core18 / base: core20 is specified, run snapcraft help [plugin name] to discover new options.

npm

The npm plugin uses npm-node-version instead of node-engine to specify the version of upstream npm to be used at build time.

e.g. wethr

 parts:
   wethr:
-    node-engine: "10.14.1"
+    npm-node-version: "10.14.1"

autotools

Autotools has migrated options from configflags to autotools-configure-parameters.

e.g. Inadyn

 parts:
   libconfuse:
    plugin: autotools
-    configflags: ['--prefix=/usr', '--disable-examples', '--disable-static']
+    autotools-configure-parameters: ['--prefix=/usr', '--disable-examples', '--disable-static']

go

The go plugin no longer requires the go-importpath to be specified. A go-channel should be specified.

e.g. slack-term

 parts:
   slack-term:
     plugin: go
-      go-importpath: github.com/erroneousboat/slack-term
+      go-channel: latest/stable

Application Definitions

Paths

Snapcraft now requires explicit paths to be specified for binaries listed in the apps stanza.

e.g. wethr

 apps:
   wethr:
-    command: wethr
+    command: bin/wethr

command-chain

Rather than specify command followed by a long list of space-separated executables, use the command-chain option to list them.

e.g. Atom

 apps:
   atom:
-    command: bin/launcher ${SNAP}/usr/share/atom/atom
+    command-chain: 
+      - bin/launcher
+    command: usr/share/atom/atom

Examples Summary

5 Likes

Just a note that --base is supported here, same goes for list-plugins

1 Like

Note: snap run --shell benefits from this.

1 Like

It’d be worth adding a case for syntax:

Failed to generate snap metadata: The specified command 'foo "&1!$(@&^%("' defined in the app 'my-app' does not match the pattern expected by snapd.
The command must consist only of alphanumeric characters, spaces, and the following special characters: / . _ # : $ -

I can’t find this in the Snapcraft docs…? I added a note to the Snapcraft Yaml reference.

2 Likes

@popey you’ve done a great job of covering a lot of lessons learned. The only thing I can think to add (at this point in time) is that i386 support is dropped, so you’ll need to do the following if your snap will/can support the other architectures:

architectures:
  - build-on: amd64
  - build-on: arm64
  - build-on: armhf
  - build-on: ppc64el
  - build-on: s390x

View this .yaml here.

4 Likes