Dependency sharing via checksums

Here’s an idea:

3 Applications depend upon file libntll.
2 of those applications use libntll 1.7, while 1 of those applications uses libntll 1.6.
libntll is distributed with each application’s package, which means there are two copies of libntll 1.7.

So, instead of having 2 copies of the same dependency, what if every time a package was updated, snapd used checksums like SHA to find copies of the same file, and then replaced those files with symlinks to one single copy stored elsewhere. When an application is updated, it checks if any changes are needed. This way, only one copy of the same file would be used, because the apps would share the same copy.

With this lifecycle:

  • When apps are installed, checksums made for every dependency
  • When a new app is added, search a checksum database for identical dependency. If found, replace them with symlinks to one copy only.
  • When the app is updated, check if dependency was modified. If so, replace the symlink with the real file, then proceed.

We have a very similar capability today via the content interface. For relevant cases, usually involving large payloads rather than an individual library, one can define a content interface and share that content across several snaps. One of the reasons we went with this instead of using a digest as suggested is that this approach allows the snap providing the content to also be updated and get serious issues fixed without having to rebuild every single dependent snap. This is an approach we’re putting in place for the gnome libraries, and it looks like kde is also following the same route.

For individual libraries and simpler cases, I wouldn’t worry so much though. The space savings are not worth the complexity cost and risk, even more in a system such as snapd that makes good use of deltas.

1 Like