Snap delta unreliable?

snapcraft
1

We have Android Studio uploaded as a snap to the store, yesterday a new version of it became available, I pushed a new update to the store and when I refresh my snap, it downloaded the complete snap of 700+ mb.

What are we doing wrong ?

Note: Android Studio is a pre-compiled tar.gz from upstream, we only redistribute it and don’t really compile it from source.

2

There are a number of things that may happen in this space, and we will fallback to using the full snap if anything at all goes wrong with the delta.

For example, the delta mechanism depends on the server having generated the right delta of course, and for a 700MB+ file it will take a little while. It will not (or should not, I haven’t seen that side of the code) block your refresh while that hasn’t taken place. Instead, it will allow you to go through and get the real file.

Another reason, which likely won’t apply to you, is that we only support deltas for a given window of revisions, otherwise we’d need to keep close to (total revisions)² deltas, which we can’t. That’s generally not an issue given automatic refreshes.

3

Is there a way we can check if there was delta produced between snaps and their size ?

Also especially for this case, can I know the delta between android-studio revision 34 and 35 ?

4

Also there probably is a bug somewhere in how build.snapcraft.io works. I only uploaded a single new commit to android-studio packaging repository BUT there are actually two new revisions.

5

The delta is meant to be completely transparent, so we intentionally don’t expose this in any way via the command line, but the code and the API are all public, so it should be easy to inspect it.

6

We just pushed a new version of Android Studio and it seems that will also cause a complete re-download of the snap. Also I am not sure if the previous build that we released a few days ago generated any delta as well.

@thomir any help here will be appreciated.

7

Hi Omer,

It appears r35 of andoid-studio was superseded by r43 in stable. We generated a download delta for this at 2018-04-11 13:31:04.063701+00. A delta between r42 and r43 was also generated at 2018-04-11 09:41:44.621912+00. Are you able to correlate these with your records? Depending on the state of the queue, it can take some time to generate a delta, so if you refreshed too quickly, the delta may not have been available yet.

In response to your earlier question, a delta between r34 and r35 was generated at 2018-03-27 09:58:35.036243+00.

8

It seems build.snapcraft is broken. If I push a new commit to the snap packaging of Android Studio, it will publish three new revisions, all for amd64.

  1. It wastes CPU cycles on the infrastructure as there is only one build needed and it creates three.
  2. Secondly and most importantly it kind of breaks delta updates.
9

Hi Omer,

Thanks for bringing this to our attention. This isn’t really delta-related, but here’s what’s happening:

Your snapcraft.yaml contains:

architectures:
  - amd64

This specifies where the snap may be run, not where it may be built. Currently, build.snapcraft.io assumes that all snaps can be built on amd64, i386 and armhf, which is why three builders are triggered for each revision. There currently is no syntax in snapcraft.yaml to specify where a snap may be built. Once we have that, build.snapcraft.io will have to be updated accordingly. There is an ongoing discussion here about how we might implement such a feature.

Note that while having multiple amd64 revisions built might be annoying for you (the developer), this won’t affect regular users since we build deltas based on what’s released to a channel, rather than what’s pushed to the store. In other words, you might have the wrong “from revision” installed (since you have access to non-released revisions), but you users can’t easily end up in that state.

Regarding download deltas, aside from some recent infrastructure issues we’ve been having, we’re not aware of any ongoing issues that can prevent deltas from being built. If you are having problems with download deltas, it helps us if you can provide the following information:

  • When you released the new revision, what the new revision number is, what the snap-id is, and what channel(s) you released to.
  • When you attempted to refresh (in UTC), what revision your client had installed (prior to refresh), what channel you were tracking, what revision the client ended up installing.
  • What architecture the client is on.

As you can see, there’s a lot we need to know in order to pin down delta generation failures. So far we’re not aware of any client-affecting delta issues, but we’re always keen to hear from users that are having problems.