Build.snapcraft.io: ARM builds spending hours in queue

snapcraft
1

Over the last couple of days I’ve noticed that armhf and arm64 builds appear to spend hours in “Building soon” purgatory before actually building.

The other architectures build in a few minutes as expected, but for some reason the ARM builds (which are what we actually need…) get stuck.

This morning I waited for a couple of hours for a build of one commit before giving up and submitting a new commit for building. That was about 45 minutes ago. This is the Launchpad page for the armhf build: https://launchpad.net/~build.snapcraft.io/+snap/908277a77436b2b858d32d6cd0bc859a-xenial/+build/391307 - currently due to “start in 46 minutes”. A few minutes ago it was due to “start in 30 minutes”, so that’s not looking like a promising trajectory.

Any ideas what’s going on?

Thanks in advance!

1 Like
2

Just wanted to bump this as it’s still an issue and causing us headaches.

A build that I triggered recently is again projected to spend at least 30+ minutes in the queue, specifically for the armhf and arm64 builds.

Should we just accept these waiting times as a fact of life these days, or is it something to look into further?

3

The Launchpad build farm

All 40 ARM builders are busy… maybe @cjwatson can shed some light on this?

4

Thanks for pointing me to that link; it’s helpful to have more insight into how the process is structured. I was concerned that was maybe there’s some kind of glitch - aside from builder resource constraints - that’s slowing things down, but looks like it’s just the latter. Interesting that the arm64 runners seem to be under so much more load than the other architectures.

1 Like
5

A certain amount of queueing is to be expected at busy times. Nothing particularly sinister is going on here - just resource constraints.

3 Likes
6

Just for the record:

%E5%9C%96%E7%89%87
%E5%9C%96%E7%89%87908×473 31.2 KB