Slow core downloads breaks tests

snapd
1

This week I’m seeing very frequent and annoying test failures that look like this:

Fetching snap "ubuntu-core"
ubuntu-core 31.77 MB / 75.46 MB   42.10% 111.01 KB/s 6m42s
<kill-timeout reached>

Note the 1Mb download speed. This is supposedly a cloud<-CDN traffic so I would expect better. Having said that, can we consider:

  • adding caching support to snapd for the sole purpose of tests, where we can drop snaps to /var/lib/snapd/cache or similar and they get used (assertions can still be online).
  • patching the test suite to download certain “heavy” snaps (like core, ubuntu-core and perhaps a few others) at most once

This would also have huge benefits for me, where I work on a metered connection all the time.

Note that I’m not proposing:

  • automatically caching downloads
  • having to solve cache invalidation problems
  • any other complexity that would be required for production use
2

We download to predictable locations, and if the file we’re trying to download already exists we resume from there. It should be possible (as long as the store doesn’t flap on whether deltas exist or not) to drop the file in place and take it from there.

For the deltas thing, we could reverse the checks and only do deltas if not resuming.

3

Thank you for the idea John! I did a quick and tiny experiment https://github.com/snapcore/snapd/pull/3641

While it does work it doesn’t seem to affect the test duration in either way. I think that whatever caused slow CDN traffic will affect this exactly the same but (hopefully) the window is smaller.