Call for testing: Github action for Snapcraft

Over the holiday I decided to teach myself to write Github Actions, and put together an action that builds Snapcraft projects.

For those who haven’t tried it out, Github Actions is Github’s built-in continuous integration and automation system. It was in beta for much of last year before being made generally available in November. It is similar to Travis CI, but allows anyone to provide canned “actions” that can be slotted into a workflow.

Using the action

Here is a trivial workflow that uses my action:

name: "build"
on: # run on pull requests and commits to master
  pull_request:
  push:
    branches:
    - master
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: jhenstridge/snapcraft-build-action@v1

The snapcraft-build-action action will do the following:

  1. fix up permissions on the root directory: for some reason it isn’t owned by root in Github’s build image, which trips up the sanity checks in snap-confine.
  2. Remove the .deb version of LXD and install the snap version. Configure LXD with lxd init --auto
  3. Install the stable version of Snapcraft
  4. Invoke Snapcraft to build the project using its LXD backend.

This should provide an environment capable of building most (or all?) snaps, including those that depend on build-snaps. It’s also complex enough that I think the action provides value over including equivalent shell commands in the workflow directly.

Integrating with other actions

The action also provides an output parameter containing the file name of the built snap. This could be used to save the snap as an artifact of the workflow:

...
    steps:
    - uses: actions/checkout@v2
    - uses: jhenstridge/snapcraft-build-action@v1
      id: snapcraft
    - uses: actions/upload-artifact@v1
      with:
        name: snap
        path: ${{ steps.snapcraft.outputs.snap }}

… or to install the snap in order to test that it actually functions as expected:

...
    - run: |
        sudo snap install --dangerous ${{ steps.snapcraft.outputs.snap }}
        # do something with the snap

Next Steps

I haven’t published the action to the Github marketplace yet. If there’s interest in this kind of thing, I thought it might be better to publish it from a more official location than my personal Github account. I think it would be worth putting together two extra actions to complement the build action:

  1. a “lint” action that runs review-tools.snap-review. This could help developers catch errors before publishing the package.
  2. a “publish” action that publishes a snap to the store. This would follow best practice, by using exported login information provided through a secret, and making sure the login info is deleted before moving on to the next step of the job.

While this overlaps a bit with the build.snapcraft.io service, I think it would be useful for projects that want more control over their CI pipeline. For instance:

  • ensure that pull requests don’t break the snap build.
  • perform extra testing steps between building the snap and publishing it.
  • publish a snap to the stable channel when a release tag is pushed to Github.
  • handle projects where the repository contains multiple Snapcraft projects, or the project isn’t located at the root of the repository.
10 Likes

Hi James,

This kind of thing is super-interesting to me. I’ve recently moved rustup's builds from travis/appveyor over to github actions, and one of the issues with that is that my nascent work to prepare rustup snaps will no longer work (they were travis based).

I found https://github.com/marketplace/actions/snapcraft-action and wonder if it might make sense to split your work into two parts – one which improves the existing action to add the lxd stuff to it, and one for the builds, leading to something like:

- uses: actions/checkout@v2
- uses: samuelmeuli/action-snapcraft@v1
  with:
    snapcraft_token: ${{ secrets.snapcraft_token}}
- uses: jhenstridge/snapcraft-build-action@v1
  id: snapbuild
- uses: actions/upload-artifact@v1
  with:
    name: snap
    path: ${{ steps.snapbuild.outputs.snap }}

If others think this is a worthwhile thing to have, I’d also suggest approaching Samuel about adopting the action into the github.com/snapcore namespace so that the actions were something along the lines of snapcore/action-prepare@v1 and snapcore/action-build@v1 you could then also have snapcore/action-upload@v1 which supported uploading to the store, with arguments for channel etc.

If you decide this sounds good and want to have a go at this kind of split-out, I’d be prepared to then try the actions out in a branch for Rustup.

I guess I didn’t see a lot of use for the action-prepare action.

If we want the actions to represent best practice, then I think having a “prepare” action that calls snapcraft login is a bad idea since that will leak the secrets to every subsequent step in the job. My thought was to make the secrets local to the “publish” action so it would essentially do:

  1. write the secret to a file in $HOME
  2. call snapcraft login --with secret-filename
  3. delete the secret file
  4. call snapcraft push to upload the snap, optionally releasing to a set of channels.
  5. call snapcraft logout

This way there wouldn’t be any secrets visible during the build, or anything that happens afterwards. It would also reduce the chance that people would upload their store credentials to Github if they aren’t actually writing a workflow that interacts with the store (e.g. if the workflow is simply checking that pull requests don’t break the snap build).

Is there anything in particular you’d want to do with a “prepare” action separate from a “build” action?

I see the setup of lxd and friends and installation of snap+snapcraft as the prepare action then. In rustup we actually end up constructing the snapcraft.yaml file on demand from scripting, so we wouldn’t expect a general build action to be able to work because there isn’t a snapcraft.yaml in the repo per-se.

Your idea for how the publish step works makes complete sense to me and I agree entirely with your reasoning.

That shouldn’t be an impediment to using my action: it will build whatever is in $GITHUB_WORKSPACE (or in a different directory if you use the path input parameter to the action). A job like the following would work, for instance:

steps:
- uses: actions/checkout@v2
- run: |
    sed `s/foo/bar` < snap/snapcraft.yaml.in > snap/snapcraft.yaml
- uses: jhenstridge/snapcraft-build-action@v1

You’ve got complete control over the workspace before you trigger the build. Or is the problem that you need the Snapcraft tool itself to generate your snapcraft.yaml file?

Re-looking at my generator I think something like your suggestion above would be fine. I’m not sure if I’d even need the --destructive given you’re managing lxd for me (though I can’t be sure since it has been a while since I tried things and can’t remember exactly what that is for), and the output variable would be enough. So I’d need your build action, and a publish action of some kind, to cover everything I had before.

1 Like

Okay, I’ve managed to get things working with your build action. Are you intending to provide a publish action?

1 Like

I’ve started on a “publish” action here:

I haven’t tagged a v1 release since it could use a bit more work (e.g. validating the input parameters to make debugging easier). You can still reference it as @master if you’re okay with potential API breaks.

A job that builds and publishes a snap might look like:

build:
  runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: jhenstridge/snapcraft-build-action@v1
      id: build
    - uses: jhenstridge/snapcraft-publish-action@master
      with:
        store_login: ${{ secrets.STORE_LOGIN }}
        snap: ${{ steps.build.outputs.snap }}
        release: edge

The store_login input parameter should be the data produced by snapcraft export-login, and release is the channel(s) where the snap should be published. If release is omitted, the action will upload the snap without publishing it.

Here’s an example run from the action’s own CI workflow: https://github.com/jhenstridge/snapcraft-publish-action/runs/385714628

1 Like

Awesome, thanks. I’ll take a look at that once I’ve finished other work tonight or else soon.

1 Like

I’ve published a v1 tag for the publish action now. The main changes were:

  • Update the documentation to suggest exporting a login with package_access permission. Things were working without it previously, but I started seeing failures today:

    Perhaps something changed in Snapcraft or the store? The package_access permission imposes a 1 year expiry date, so this also means the login data will need to be replaced yearly.

  • Fail fast if the login data is empty. This is most likely to happen if you try to use the action in a workflow triggered by pull requests, where secrets are not made available.

  • Fail fast if the snap file can not be found.

I’ve also updated the build action so it folds the output lines related to installing Snapcraft itself, so that the project build output is more prominent.

2 Likes

I shall endeavour to update to this published action in my PR early in the week. Thanks James.

1 Like

@sergiusens: what needs to happen in order to publish these actions on the snapcore project? Since these are related to Snapcraft rather than snapd, I was pointed towards you as a decision maker.

The rustup developers would like to use the actions (see rustup PR #1898), but bring up reasonable concerns about trusting actions published by me personally. The snapcraft-publish-action in particular deals with Snap Store credentials, which is not the kind of thing we want to teach developers to reveal to random people. The snapcraft-build-action doesn’t deal with secrets, but users of the action are still trusting that it doesn’t tamper with the build process.

bonus points if you also add optional review-tools checking of the built snap before uploading to the store ;-p

2 Likes

I think that might be better as its own action, or as a feature of the build action. If I’m writing a workflow to test pull requests, I can’t easily use the publish action because it won’t have access to secrets.

I’d still be interested in the linter warnings/errors from review-tools: especially if they can be sensibly surfaced in the pull request.

1 Like

Good point on making it a separate action :slight_smile: I concur!

https://www.youtube.com/watch?v=i5j1wWY-qus

1 Like

yoink

3 Likes

Let’s please set this up in our upcoming sprint.

4 Likes

I’d just like to add thanks to all concerned, these actions will really make it easier for me to try and progress the rustup snap story.

2 Likes

I just test, if it’s possible to build a snap, with the github runner for local building tasks.

@sergiusens without wanting to sound like I’m rushing you – do you know when this might be?