Allow zstd compression for snaps

It should be possible to use ZSTD compression for snaps. It offers better compression/performance ratio. I do understand the reason for not exposing that currently because systems running older kernel may not support ZSTD (that’s understandable).

Since Snap store only support xz and lzo compressed snaps, I don’t see any hard reason in snapcraft not supporting that for the snap archive. This can be useful for custom snap stores, where we “know” on what hardware our apps are going to run.

Maybe expose that behind an experimental flag ?

2 Likes

Also: Can we please revisit the default compression format again. Especially is snapd still running on distros (and Canonical customer devices) that are pre-linux 4.14 ?

My proposal about using desync for Delta updates also suggests that only a single instance of a snap should live on storage AND delta chunks for older revisions should live on storage and snap revert should reconstruct the old revision using those chunks, instead of “wasting” disk space as we have today.

3 Likes

Yes we have devices with the 4.4 kernel out there, as well as devices on Centos which use some 3.X kernel IIRC.

1 Like

ping @sergiusens, what are your thoughts on this ? Would something like that be easy to achieve ?

We could probably support ZSTD, but bring in a warning when it’s used. In a place like this https://github.com/snapcore/snapcraft/pull/3421/files

1 Like

Did some investigation into this. It seems snapcraft ultimately calls snap pack to create the archive, which currently only supports xz and lzo.

This extends the scope of this topic beyond snapcraft (the tool).

@ijohnson would you accept a patch that adds zstd support to snap pack ?

1 Like

The issue with enabling this is that allowing snaps to be uploaded with compression formats that are not universally supported like XZ and LZO introduces the possibility of folks uploading snaps that cannot be used on platforms that we, Canonical and the snapd team, specifically support.

You have mentioned a custom snap store, if you have a brand store I would suggest reaching out to your sales rep folks about the possibility of enabling this for your brand store, it may be something we can do for specific brand stores until we have a more general solution in place which allows us to provide a better user experience / story for users on distros which may not support new compression formats like zstd, etc. See my explanation here for more details: https://ubuntu.com/blog/why-lzo-was-chosen-as-the-new-compression-method

Regarding the patch to snapd, I’m sure it’s extremely simple, but we can’t accept it until we have a better story for users on old distributions that don’t support the new formats.

We could easily probably add support for GZIP and “none” compression methods since those should be easily supported everywhere but indeed the more interesting thing is ZSTD which brings the best of both worlds for both size and speed.

I believe @om26er was referring to a non-Canonical Snap Store when they mention “custom” - i.e. not the Public store, and not a Brand store. They’re working on reimplementing the Snap Store API in an open source implementation over on GitHug.

True, @ijohnson , RHEL/CentOS 7 is widely adopted, will be maintained until 2024 and uses 3.10 (heavily patched, but still) .

See this thread I created yesterday, by coincidence.

If you attempt to limit what the capability of the snap project is to achieve an arbitrary amount of backward compatibility, this project shall stagnate. Addition of modules that allow for compatibility should be the most ideal method of ensuring compatibility, but if they hinder progress, I shall greatly support their removal. Different configurations of GNU/Linux should be what this project is attempting to support rather than old packages, because the latter is what has caused Windows to become as bloated as it is, and shall divert precious resources to something that is known to be incredibly difficult to maintain whilst performing improvement.

For what it is worth, if you had a suitably patched implementation of snap pack, it would be possible to have an unpatched Snapcraft build such a snap with the following in the snapcraft.yaml file:

passthrough:
  compression: zstd

This won’t generate a specific warning, but it does signal “I know what I’m doing isn’t ready for prime time”.

There are checks for the compression method in the automated review tool which don’t rely on “snap pack”, so in theory the snapd change could be accepted without risk of zstd snaps being published to the store before the compatibility issues have been evaluated.

1 Like

That is true and also why I mentioned that in my initial comment that, there is no risk in a developer publishing their snaps using wrong compression, because the store does not allow that.

We could also make zstd support hidden i.e. by keeping snap pack --help unchanged.

The store implementation that I have is currently not open-source but I will do that eventually. We are using that in our company internally.

The basic architecture is:

  • Patched snapd to support custom signing authority
  • Django based web service, that signs snaps (creates assert file) and uploads to S3
  • A cli tool (written in python) that talks to snapd sock but is also able to talk to our django based service to install/update snaps (our store takes precedence over Snap store)
  • The service also supports delta updates using desync
  • We also have our implementation of “channels”.

Aah, ok, my mistake. I recall someone was working on an open snap store implementation on GitHub and had erroneously associated that detail with you :slight_smile:

Edit: it was Futuretim: Announcing Project Kebe (open-source Snap store start) - snapd - snapcraft.io

In general we try not to add features partially like supporting this new compression method without having a plan for the rest of the feature as then it results in us maintaining a bunch of half-finished features which is not productive and we already have this problem too much, so I would be -1 to a patch to snapd allowing zstd until we have more of a plan in place unfortunately.

1 Like