To build the Certbot snaps maintained by the “Certbot Project” we make heavy use of
snapcraft remote-build. One problem we’ve seen with increasing frequency lately are builds indefinitely hanging in the “Currently building” state. When this happens, the build log shown on launchpad always looks like this.
We saw this problem for the first time back in January, but have seen it 3 times in the last week. We build on amd64, arm64, and armhf and have seen the problem on both ARM architectures. We haven’t seen it on amd64 (yet).
What can we do to fix this problem? Is this a bug in snapd/launchpad that I should report or track elsewhere?
Another option that’d help us would be to add a timeout option to
snapcraft remote-build to have it delete the builds after this timeout has been reached if something like this is supported by launchpad’s API. Once our builds get in this state, they remain stuck there and future runs of
snapcraft remote-build will reconnect to them until our source code changes or we manually delete the build off launchpad. I do not believe
snapcraft remote-build's current
--timeout option works for us because its help text says “Note that the build will continue on Launchpad and can be resumed later.”
Any help in fixing this problem is much appreciated.