This is an initial draft subject to change
Introduction
snapcraft has always assumed that the build environment the developer was triggering the build from was the appropriate one for the snap at run-time.
This assumption has many fallacies from the point of view of the current environment. It assumes the association with what is called the core
snap today is well known and that it is based out of Ubuntu 16.04LTS. Furthermore, when building on an environment different than Ubuntu 16.04LTS, there is no clear indication that the built snap may or may not work against this core
base.
With the introduction of bases, which spreads the combinations of appropriate build environments for the relevant bases, a new problem is inserted into the problem domain to solve, this latter problem leads to snapcraft having to learn how to gain knowledge on how to operate in all these disparate environments to construct a snap that shall work with the desired base.
Some of these problems could be solved by extending the current implementation of cleanbuild
and broadly extend and productize the use of the persistent project container support, which in itself would be limited by the fact that we want to be able to target multiple bases, using the appropriate build environment triggered from different host operating systems (e.g.; trigger a build of a snap that targets core18
which would build on Ubuntu 18.04LTS from an Arch Linux based host), use of containers could limit the use of certain features of snapcraft, such as use of build-snaps
.
The current implementation would put the burden on snapcraft itself on how to operate with the given build environment; snapcraft has in many places assumptions that it is working with an environment that looks like Ubuntu. Such is the case that plugins work on the assumption that Ubuntu package names would be available.
Proposal
The gist of the proposal is to decouple snapcraft’s environment setup from snapcraft’s core logic and the plugins themselves. The mechanisms to do so shall be handled through hooks snapcraft would call into to provision the environment appropriately.
To be able to build from anywhere to anywhere, the proposal is to make use of virtual machine images created specifically to target the base. They will be downloaded and run for a given snapcraft project depending on the base
keyword inside snapcraft.yaml
.
Backwards compatibility
snapcraft will retain the current behavior it currently carries as long as base
is not explicitly specified in snapcraft.yaml
, providing a path forward for those using bases.
Build Environment
The build environment shall be responsible for determining a list of things:
- how to install
build-packages
. - how to download and unpack
stage-packages
- if
build-snaps
are supported. - how the environment needs to be setup for each plugin to function correctly.
- the matching snap architecture string for a given architecture used (e.g.; to support use of
on <arch>
). - the architecture triplet to use.
- Reusable components provided by the
base
such as where a sane cut-off can be made forstage-packages
and files that can be reused from such base).
The proposal is that each build environment shall have hooks
snapcraft can execute to achieve these goals.
The environment shall be provided in a build-environment.yaml
file at the index of the Virtual Machine Image. Attributes of the provided build-environment.yaml
can be overridden by providing a top level build-environment
entry in snapcraft.yaml
.
The structure is as follows:
build-environment:
arch-triplet: <architecture-triplet>
build-packages-handler: <path-to-exec-to-handle-build-packages>
stage-packages-handler: <path-to-exec-to-handle-stage-packages>
plugin-<plugin-name>-setup: <path-to-exec-to-handle-setup-for-plugin>
classic-setup: <path-to-exec-to-handle-setup-for-classic>
Virtual Machine Image Indexing
Virtual machines shall be provided for any base
snapcraft is to support. These Virtual Machine Images will be indexed in snapcraft’s code base as a phase 0 approach for rapid delivery of the solution, with a farther looking goal of having some service where owners of a certain base
can upload newer versions of their virtual machine images for consumption.
snapcraft will minimally load snapcraft.yaml
, verify the value of base
and find the matching Virtual Machine Image that can build for that base, the mapping should be 1:1 but up to the provider of the actual base.
Upon finding a match, snapcraft shall retrieve the image from the URL specified in the index and once download is complete, verify its validity against an indexed sha3-384 hash. If there are no matches, snapcraft shall gracefully fail.
snapshots of the Virtual Machine Images in a booted state shall be provided for different CPU and RAM configurations.
The index shall be yaml encoded with the following structure,
bases:
<base>:
<snap-arch>:
url: <link-to-base-vmi>
sha3-384: <sha3-384-hash-for-base-vmi>
revision: <optional-revision-of-the-vmi>
snapshots: <list-of-snapshots-for-different-configurations>
As an example,
bases:
core18:
amd64:
url: https://cdimage.ubuntu.com/snaps/core18.vm
sha3-384: <sha3-384-for-core18.vm>
arm64:
url: https://cdimage.ubuntu.com/snaps/core18.vm
sha3-384: <sha3-384-for-core18.vm>
core:
amd64:
url: https://cdimage.ubuntu.com/snaps/core.vm
sha3-384: <sha3-384-for-core.vm>
fedora29:
amd64:
url: https://fedoraproject.org/snaps/fcore29.vm
sha3-384: <sha3-384-for-fcore29.vm>
revision: 1
revision
being optional, indicating in the case of fedara29
that the client needs to download a newer revision in case of already having the first one (the default is 0).
snapcraft will need to have a translation layer for local architectures reported by the OS to that of
<snap-arch>
.
Executing a build
When running snapcraft
, the Virtual Machine Image shall be downloaded with its corresponding snapshots if provided. After launching the VM, snapcraft will mount the project inside the virtual machine instance and,
Using the host
snapcraft will detect when it is running inside LXD or docker and not trigger the behavior of launching a Virtual Machine to execute the build. If a build-envionment.yaml
is provided, snapcraft will make use of it, if not it assumes the environment is already correctly setup to construct the snap.
These are considered specialized environments, if auto-detection fails or is not supported, snapcraft shall provide an option --use-host
to trigger building on the host snapcraft is invoked on (i.e.; actual host, VM or container).
Current Opens
- Indexing:
- shall we add a requirement to have GPG signatures for the Virtual Machine Image or its hash?
- is
<snap-arch>
what we want to index on? It would provide a transparent path towards adding these images to the Snap Store.