Concurrent snapcraft builds; is it safe?

snapcraft
1

Hey there,

We are trying to do snapcraft builds in CI pipelines. As of now we have gitlab shell runners in a persistent VM, which means that every snap build runs on the same machine.

  1. Is it safe to run multiple instances of snapcraft on the same machine? For the same snap?

Inspecting LXD we see that there remain several stopped containers after the build has stopped:

gitlab-runner@clp:~$ lxc ls --project=snapcraft
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
|                             NAME                              |  STATE  | IPV4 | IPV6 |   TYPE    | SNAPSHOTS |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| snapcraft-conf-hardware-on-amd64-for-amd64-531736       | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| snapcraft-conf-hardware-on-amd64-for-amd64-531737       | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| snapcraft-conf-hardware-on-amd64-for-amd64-574559       | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| snapcraft-pc-on-amd64-for-amd64-568438                  | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| snapcraft-pc-on-amd64-for-amd64-779335                  | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| snapcraft-pc-on-amd64-for-amd64-1597995                 | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| snapcraft-pulse-server-on-amd64-for-amd64-1294914       | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
| snapcraft-pulse-server-on-amd64-for-amd64-1296058       | STOPPED |      |      | CONTAINER | 0         |
+---------------------------------------------------------------+---------+------+------+-----------+-----------+
  1. It seems that if there is two concurrent build jobs snapcraft starts a new container. Is this correct?
  2. How does the postfix number get decided?
  3. Is there any way to tell snapcraft what you want the name of the container to be?
  4. How do we do proper cleanup for these containers? Can we remove them completely after a CI job has finished?
  5. Any other tips on how to do this properly and cleanly?

Kind regards, Charlee

2
  1. Is it safe to run multiple instances of snapcraft on the same machine?

Yes.

For the same snap?

You can build multiple snaps with the same name, but the builds need to be in different project directories.

You could experiment with concurrent builds in the same project directory for different architectures with the --build-for or --platform flag but this is not a supported or tested behavior.

  1. It seems that if there is two concurrent build jobs snapcraft starts a new container. Is this correct?

A build environment is specific to:

  • the snap name
  • the snap base
  • the build-on architecture
  • the build-for architecture
  • the project directory

So no, currently running builds do now affect if a new build environment is created.

  1. How does the postfix number get decided?

The postfix number 1597995 in examples like snapcraft-pc-on-amd64-for-amd64-1597995 is the inode of the project directory.

However, LXD has a 63 character maximum for container names. If the name of the snap is too long, this maximum will be exceeded and snapcraft will have to hash the name, truncate the name, and append the hash.

It will be formatted as:

<truncated-name>-<hash-of-name>
└     1 - 40   ┘1└     20     ┘
  1. Is there any way to tell snapcraft what you want the name of the container to be?

No.

  1. How do we do proper cleanup for these containers?

This is a often requested feature that is being discussed here: add `charmcraft purge` or `charmcraft cleanall` command to clean all instances · Issue #1042 · canonical/charmcraft · GitHub. Even though that is in the charmcraft repo, the design and implementation in charmcraft will be shared with snapcraft.

Can we remove them completely after a CI job has finished?

Yes. If the CI instance persists across runs, you can keep the base-instance- containers to speed up future builds. If not, you can remove everything.

  1. Any other tips on how to do this properly and cleanly?

You can run snapcraft clean after building to remove containers for that build.

To clean up everything, the recommended way is to use a simple bash or python script to remove the with LXD. Something like this should work fine:

lxc --project snapcraft list --format csv -c n | xargs -I {} lxc --project snapcraft delete --force {}
3

Thanks for the detailed reply :slight_smile:

We are doing snapcraft builds in gitlab jobs on a VM. Gitlab creates multiple project directories if there is concurrent jobs for the same project, so I guess that ensures that the containers are different.

Thanks a lot!

1 Like