Breaking up snap into inter-dependent parts

I’d like to get the community’s ideas on how to best approach a sort of optimization we have in mind.

We currently have a monolithic snap that includes some Python code, Python binaries + libraries, NodeJS + libraries, Redis, and a few other packages. One of the beauties of the snap ecosystem is that it’s possible to precisely define and package together a set of inter-dependent components like this. Indeed, snaps appear to be monolithic by design.

However in reality, from one revision to another, 99% of the snap’s contents remain static. In our case, while we push updates to our own Python code quite frequently (sometimes multiple times per week), most of the underlying binaries and libraries barely ever change (maybe once a year, if that).

Therefore we’d like to explore a way to split our snap into a “my-core” snap - containing the stuff that is pretty static - and an “my-app” snap - containing the stuff that changes frequently. I suspect this is not an uncommon scenario.

A similar question was brought up in Specifying other snaps as dependency. The answer appears to be that explicit dependencies are currently not possible, though I feel like this can be addressed otherwise (more on that below). To be clear, I’m not talking about adding the “my-core” snap as a dependency in “my-app” snapcraft.yaml so that “my-app” includes “my-core”. What I have in mind is two snaps installed side-by-side, so that we can capture the following benefits:

  • Building new releases of the “my-app” snap takes minimal time (e.g. we’re not building Redis from scratch every time)
  • Deploying new releases of the “my-app” snap takes minimal bandwidth. At the moment, deltas between releases of the monolithic snap can be many MB, even for minor code changes. This matters since most of the devices we’re deploying to are in very bandwidth-constrained environment.

This brings up three questions:

  1. Is this a sensible/practical idea, or are we setting ourselves up for a world of pain?
  2. Are there any good examples where this has been done for other snaps, or guidelines on how to do it “well”? I suspect that the way to implement this is to have the “my-core” snap expose binaries and libraries via the content interface, and for the “my-app” snap to consume and run these. I’d love to see if/how this has been done elsewhere and learn from others’ experience.
  3. Is there a reasonably elegant way to manage dependencies between versions of “my-core” and “my-app”? In an ideal world, we would be able to say that installing “my-app” automatically triggers an install of “my-core”, with some revision dependency definition. It looks like doing that, specifically, is not possible - but from what I’ve seen the gating mechanism described in https://docs.ubuntu.com/core/en/build-store/refresh-control may offer an appropriate level of dependency control. E.g. let’s say we have my-core-v1 and my-core-v2, and my-app revision 20 and above are incompatible with my-core-v1, and require my-core-v2 in order to function. Then we can set my-core-v1 as a gating snap for my-app revisions up to 19 (but not above), and my-core-v2 as a gating snap for higher revisions of my-app. How the upgrade path from my-core-v1 to my-core-v2 is handled is I suppose a separate exercise. Would something like that be sensible, or am I barking up the wrong tree?

Thanks in advance for any ideas!

2 Likes

I think that’s a very sensible approach. E.g. gnome-3-34-1804 is consumed via the content interface and it is very commonly used for gtk/gnome apps.

Meaningful versioning (see the gnome platform snap) is a must, of course.

1 Like

Another approach may be to keep including everything in a single snap, but don’t build the seldom-changing code through this snapcraft.yaml: just add the libs to the snap, having built them separately. So, you manage the lib dependencies, and binary delta upgrades will presumably keep upgrades of the snap small, and build time decreases dramatically.

1 Like