The problem
Currently, due to the limitation of the Snap Store build service and Launchpad, it’s not possible to ensure that a snap will be built on certain tagged release as a build is only triggered when:
- Someone pushes to the associated repository(which development tip (a.k.a.
HEAD
) not necessarily a tagged release) - A Launchpad import occurs if the repo is set to be importing periodically from an external repository
The workaround
The selective-checkout
scriptlet enhances the pull step to only build development snapshots snaps if the latest tagged release has already been promoted to the stable channel. This ensures that there’s always a revision of the stable release snap available in the edge channel for the publisher to promote to the stable channel.
Case: Tagged release published to the stable channel
selective-checkout: Last tagged release(1.8.1) is in the stable channel, building development snapshot
selective-checkout: Snap version determined to be "1.8.1-26-gd6ddb74".
Case: Tagged release hasn’t been published to the stable channel
selective-checkout: Last tagged release(1.8.2) hasn't promoted to the stable channel(1.8.1) on the Snap Store, building tagged release instead.
Note: checking out 'v1.8.2'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at dcd1bd3 release version 1.8.2
selective-checkout: Snap version determined to be "1.8.2".
This scriptlet is inspired by the snapcrafters’ adopted snaps workflow, documented in the following forum topic:
Refer to the following forum topic for a possible solution proposal to this problem:
Prerequisites
- Your snap recipe is using the
base
keyword (i.e. not the legacy snapcraft syntax) - Your main component part must be using one of the following version control systems:
- Git
- Mercurial
- Subversion
- The upstream release tags must be version sorted and be either dot-separated or underscore-separated, optionally prefixed with a fixed string, e.g.
1.2
,1.2.3
,v1.2.3
, orFOOBAR_3_0_0
. - The build host must have internet access (for querying snap status on the snap store)
- Your snap must not be private on the snap store (
unlisted
is fine)
How to use
-
Download the release package from the project’s Releases page.
-
Extract the release package.
-
Copy the scriptlets/selective-checkout scriptlet file from the extracted release package files to your project’s source tree. The recommended location to store the file is /snap/local/scriptlets.
NOTE: It is recommended to document the version of the scriptlet somewhere(e.g. in the commit message) to track its version.
-
In the top-level snap metadata, DO NOT SET the
version
key, instead, set theadopt-info
key with the main component part’s name as the value.#version: adopt-info: your-main-part
-
In your main component part, merge the following part definition:
parts: your-main-part: # If you specify the `source-depth` property should be deep enough to include a tagged release #source-depth: 500 build-packages: # selective-checkout dependencies - curl - jq - sed # Uncomment the VCS your main part is using #- git #- mercurial #- subversion override-pull: | set -o nounset # For core22 or newer bases: if ! craftctl default; then printf \ 'Error: Unable to run the default action of this step.\n' \ 1>&2 exit 2 fi # For core20 and older bases: if ! snapcraftctl pull; then printf \ 'Error: Unable to run the default action of this step.\n' \ 1>&2 exit 2 fi project_dir="${CRAFT_PROJECT_DIR:-"${SNAPCRAFT_PROJECT_DIR}"}" if ! "${project_dir}/scriptlets/selective-checkout"; then printf \ 'Error: Unable to run the selective-checkout scriptlet.\n' \ 1>&2 exit 2 fi # Do your additional regular stuff
Known issues
Currently, there’s a bug in one of Snapcraft’s dependencies that prevents selective-checkout
from working on core24 snaps: Set the project environment in the Lifecycle Service · Issue #320 · canonical/craft-application
The logic
- The scriptlet will checkout the latest tagged stable release and build tagged release snap if the latest tagged release hasn’t been promoted to the stable channel (check is based on version-sort)
- The scriptlet will checkout the latest tagged beta/candidate releases if they were found and it is version-sortwise more recent than the version shipped in the more stabler channels (for releasing the snap to beta/candidate channels)
- The scriptlet will not touch the source tree and build as-is if
- A VCS repository is not found (will set the snap version to
unknown
) - Working tree is dirty(building test snaps)
- A VCS repository is not found (will set the snap version to
Supported command-line options
--append-packaging-revision
Appending the packaging revision string after the snap version string, in the format: _upstream_version_+pkg-_packaging_revision_
. Useful for out-of-upstream-source-tree snap build.
--beta-tag-pattern=_pattern_
Customize the extended regular expression pattern of the beta release tags.
Default: -beta[[:digit:]]+$
Example: b[[:digit:]]+$
--debug
(DEPRECATED)
Enable debugging messages. This command option is only supported for compatibility reasons, see the SELECTIVE_CHECKOUT_DEBUG
environment variable for the recommended method.
--debug-tracing
Enable execution tracing, just in case you’ve encountered a bug.
--dry-run
Don’t run snapcraftctl
so that you can run it out of snap build(after changing the working directory to SNAPCRAFT_PART_SRC
and exporting the SNAPCRAFT_PROJECT_NAME
environment variable, of course!)
--force-snapshot
Force building development snapshot regardless of the status of the snap release, for whatever reason.
--force-stable
Force building stable release regardless of the status of the snap’s releases, for whatever reason.
--packaging-dirty-marker-postfix=_postfix_
The postfix string to be appended to the packaging revision as the dirty marker.
Default: -d
--packaging-revision-minimal-length=_string_length_
The minimal length of the packaging revision string.
Default: 4
--release-candidate-tag-pattern=_pattern_
Set the extended regular expression pattern to match the beta release tags.
Default: -rc[[:digit:]]+$
--release-tag-pattern=_pattern_
Set the extended regular expression pattern of all to release tags to filter from all tags, useful when non-release tags are used.
Default: .*[._].*
(any string that contain a dot(.
) or an underscore(_
)
Example: foobar_[[:digit:]]+(\.[[:digit:]]+){2}
--release-tag-prefix=_prefix_
Set the prefix for all release tags, the prefix will be stripped from the resulting snap’s version string.
Default: v
--snap-postfix-seperator=_seperator_
Set the separator for the postfixed string in the snap version string, the postfixed string will be stripped from the snap version string before comparing with the stripped upstream release version.
Default: +
--stable-tag-pattern=_pattern_
Customize the extended regular expression pattern of the stable release tags.
Default: (unset) Derived from the exclusion of all the release candidates and beta release tags.
--upstream-dirty-marker-postfix=_postfix_
The postfix string to be appended to the upstream version string as the dirty marker.
Default: -dirty
--upstream-revision-minimal-length=_string_length_
The minimal length of the upstream revision string.
Default: 7
Supported environment variables
SELECTIVE_CHECKOUT_DEBUG
Enable debugging messages, supported value: true
/ false
Default: false
The implementation
林博仁 Buo-ren, Lin / The selective-checkout override-pull scriptlet · GitLab
Snaps that are using this stage snap
- mikf/gallery-dl: Command-line program to download image-galleries and -collections from several image hosting sites - patch
- Git : Code : Unofficial Snap Packaging for LÖVE : 林博仁(Buo-ren, Lin) - patch
- ImageOptim/gifski: GIF encoder based on libimagequant (pngquant). Squeezes maximum possible quality from the awful GIF format. - patch
- nicolargo/glances: Glances an Eye on your system. A top/htop alternative for GNU/Linux, BSD, Mac OS and Windows operating systems. - patch
TODO
Patches welcome!
- Support other SCMs(Bazaar, etc.)
- Don’t require user exporting
SNAPCRAFT_PROJECT_NAME
before running--dry-run
Reference
The following materials are referenced during the development of this project:
- Using the craftctl tool - doc - snapcraft.io
For the new syntax of customizing the step’s behaviors. - Part “after” changed behavior - snapcraft - snapcraft.io
For the new behavior of theafter
part definition stanza in Snapcraft version 7. [/quote]
The problem
Currently, due to the limitation of the Snap Store build service and Launchpad, it’s not possible to ensure that a snap will be built on certain tagged release as a build is only triggered when:
- Someone pushes to the associated repository(which development tip (a.k.a.
HEAD
) not necessarily a tagged release) - A Launchpad import occurs if the repo is set to be importing periodically from an external repository
The workaround
The selective-checkout
scriptlet enhances the pull step to only build development snapshots snaps if the latest tagged release has already been promoted to the stable channel. This ensures that there’s always a revision of the stable release snap available in the edge channel for the publisher to promote to the stable channel.
Case: Tagged release published to the stable channel
selective-checkout: Last tagged release(1.8.1) is in the stable channel, building development snapshot
selective-checkout: Snap version determined to be "1.8.1-26-gd6ddb74".
Case: Tagged release hasn’t been published to the stable channel
selective-checkout: Last tagged release(1.8.2) hasn't promoted to the stable channel(1.8.1) on the Snap Store, building tagged release instead.
Note: checking out 'v1.8.2'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at dcd1bd3 release version 1.8.2
selective-checkout: Snap version determined to be "1.8.2".
This scriptlet is inspired by the snapcrafters’ adopted snaps workflow, documented in the following forum topic:
Refer to the following forum topic for a possible solution proposal to this problem:
Prerequisites
- Your snap recipe is using the
base
keyword (i.e. not the legacy snapcraft syntax) - Your main component part must be using one of the following version control systems:
- Git
- Mercurial
- Subversion
- The upstream release tags must be version sorted and be either dot-separated or underscore-separated, optionally prefixed with a fixed string, e.g.
1.2
,1.2.3
,v1.2.3
, orFOOBAR_3_0_0
. - The build host must have internet access (for querying snap status on the snap store)
- Your snap must not be private on the snap store (
unlisted
is fine)
How to use
-
Download the release package from the project’s Releases page.
-
Extract the release package.
-
Copy the scriptlets/selective-checkout scriptlet file from the extracted release package files to your project’s source tree. The recommended location to store the file is /snap/local/scriptlets.
NOTE: It is recommended to document the version of the scriptlet somewhere(e.g. in the commit message) to track its version.
-
In the top-level snap metadata, DO NOT SET the
version
key, instead, set theadopt-info
key with the main component part’s name as the value.#version: adopt-info: your-main-part
-
In your main component part, merge the following part definition:
parts: your-main-part: # If you specify the `source-depth` property should be deep enough to include a tagged release #source-depth: 500 build-packages: # selective-checkout dependencies - curl - jq - sed # Uncomment the VCS your main part is using #- git #- mercurial #- subversion override-pull: | set -o nounset # For core22 or newer bases: if ! craftctl default; then printf \ 'Error: Unable to run the default action of this step.\n' \ 1>&2 exit 2 fi # For core20 and older bases: if ! snapcraftctl pull; then printf \ 'Error: Unable to run the default action of this step.\n' \ 1>&2 exit 2 fi project_dir="${CRAFT_PROJECT_DIR:-"${SNAPCRAFT_PROJECT_DIR}"}" if ! "${project_dir}/scriptlets/selective-checkout"; then printf \ 'Error: Unable to run the selective-checkout scriptlet.\n' \ 1>&2 exit 2 fi # Do your additional regular stuff
The logic
- The scriptlet will checkout the latest tagged stable release and build tagged release snap if the latest tagged release hasn’t been promoted to the stable channel (check is based on version-sort)
- The scriptlet will checkout the latest tagged beta/candidate releases if they were found and it is version-sortwise more recent than the version shipped in the more stabler channels (for releasing the snap to beta/candidate channels)
- The scriptlet will not touch the source tree and build as-is if
- A VCS repository is not found (will set the snap version to
unknown
) - Working tree is dirty(building test snaps)
- A VCS repository is not found (will set the snap version to
Supported command-line options
--append-packaging-revision
Appending the packaging revision string after the snap version string, in the format: _upstream_version_+pkg-_packaging_revision_
. Useful for out-of-upstream-source-tree snap build.
--beta-tag-pattern=_pattern_
Customize the extended regular expression pattern of the beta release tags.
Default: -beta[[:digit:]]+$
Example: b[[:digit:]]+$
--debug
(DEPRECATED)
Enable debugging messages. This command option is only supported for compatibility reasons, see the SELECTIVE_CHECKOUT_DEBUG
environment variable for the recommended method.
--debug-tracing
Enable execution tracing, just in case you’ve encountered a bug.
--dry-run
Don’t run snapcraftctl
so that you can run it out of snap build(after changing the working directory to SNAPCRAFT_PART_SRC
and exporting the SNAPCRAFT_PROJECT_NAME
environment variable, of course!)
--force-snapshot
Force building development snapshot regardless of the status of the snap release, for whatever reason.
--force-stable
Force building stable release regardless of the status of the snap’s releases, for whatever reason.
--packaging-dirty-marker-postfix=_postfix_
The postfix string to be appended to the packaging revision as the dirty marker.
Default: -d
--packaging-revision-minimal-length=_string_length_
The minimal length of the packaging revision string.
Default: 4
--release-candidate-tag-pattern=_pattern_
Set the extended regular expression pattern to match the beta release tags.
Default: -rc[[:digit:]]+$
--release-tag-pattern=_pattern_
Set the extended regular expression pattern of all to release tags to filter from all tags, useful when non-release tags are used.
Default: .*[._].*
(any string that contain a dot(.
) or an underscore(_
)
Example: foobar_[[:digit:]]+(\.[[:digit:]]+){2}
--release-tag-prefix=_prefix_
Set the prefix for all release tags, the prefix will be stripped from the resulting snap’s version string.
Default: v
--snap-postfix-seperator=_seperator_
Set the separator for the postfixed string in the snap version string, the postfixed string will be stripped from the snap version string before comparing with the stripped upstream release version.
Default: +
--stable-tag-pattern=_pattern_
Customize the extended regular expression pattern of the stable release tags.
Default: (unset) Derived from the exclusion of all the release candidates and beta release tags.
--upstream-dirty-marker-postfix=_postfix_
The postfix string to be appended to the upstream version string as the dirty marker.
Default: -dirty
--upstream-revision-minimal-length=_string_length_
The minimal length of the upstream revision string.
Default: 7
Supported environment variables
SELECTIVE_CHECKOUT_DEBUG
Enable debugging messages, supported value: true
/ false
Default: false
The implementation
林博仁 Buo-ren, Lin / The selective-checkout override-pull scriptlet · GitLab
Snaps that are using this stage snap
- mikf/gallery-dl: Command-line program to download image-galleries and -collections from several image hosting sites - patch
- Git : Code : Unofficial Snap Packaging for LÖVE : 林博仁(Buo-ren, Lin) - patch
- ImageOptim/gifski: GIF encoder based on libimagequant (pngquant). Squeezes maximum possible quality from the awful GIF format. - patch
- nicolargo/glances: Glances an Eye on your system. A top/htop alternative for GNU/Linux, BSD, Mac OS and Windows operating systems. - patch
TODO
Patches welcome!
- Support other SCMs(Bazaar, etc.)
- Don’t require user exporting
SNAPCRAFT_PROJECT_NAME
before running--dry-run
Reference
The following materials are referenced during the development of this project:
- Using the craftctl tool - doc - snapcraft.io
For the new syntax of customizing the step’s behaviors. - Part “after” changed behavior - snapcraft - snapcraft.io
For the new behavior of theafter
part definition stanza in Snapcraft version 7.