Existing snaps that produce i386 builds are supportable for the lifetime of Ubuntu 16.04 LTS or Ubuntu 18.04 LTS when using the core or core18 snaps as the base. Publishers who want to move to ‘base: core20’ must drop builds for the i386 architecture since it isn’t unavailable.
This presents a problem for publishers whose core/core18 builds are using the latest track and they want to move latest to core20. In my case, I wanted to continue to support i386 for the lifetime of Ubuntu 18.04 LTS, so I:
created a non-latest track that will use ‘base: core18’ and snapcraft released appropriate revisions from latest into it
created a branch of my project that continues to use ‘base: core18’ and push auto-builds (that includes i386) of this branch to the non-latest track (with master continuing to be pushed to ‘latest/edge’)
updated ‘master’ to use ‘base: core20’ and drop i386 from the list of auto-builds
This works well enough organizationally but there are some problems:
existing i386 users cannot be automatically migrated to the non-latest track
new i386 users can’t use ‘snap install foo’ and must discover they should use the channel (that could be solved via documentation)
the publisher needs to decide if it is better to close the i386 arch on latest or release non-latest/i386 builds to latest
In order to best support users, I think the best choice is to push the non-latest/i386 builds to latest so they at least get bug fixes, but this is awkward for the publisher and having a divergent codebase for i386 in latest is not optimal. Not to mention, different archs will have different support lengths within the same track.
Perhaps we should consider a mechanism to route arches to different tracks to handle this sort of thing?
You might be able to make this work without special tracks by using epochs.
I believe each snap is using epoch 0 by default, but you can publish a core18 build for all architectures (including i386) to stable explicitly using epoch 0 just in case. Then publish another core18 set for all architectures EXCEPT i386 using epoch 0*. Finally, publish a core20 set for all architectures (which at this point excludes i386) using epoch 1.
Once you get to that point, any time you update epoch 1 you’re updating core20/non-i386 folks. Any time you update epoch 0 you’re only updating core18/i386 folks. No track switching or snap install foo breakage involved. Theoretically.
Different developers are going to choose different paths. For distro-ish snaps a viable i386 version needs to be maintained, but some ISVs might be in a position to choose to drop an architecture completely, for example. I see a few major paths, of which I think you describe 3.2, though I’d be interested to know why you didn’t consider option 1.
Current version, old base: The best option for users, possibly the only sensible option for distro-ish snaps, snaps that are tied to a fast-moving online service, or snaps that have significant security surface.
latest/stable: Only option that makes sense. But slightly confusing for users if they care about the underlying base.
No version: The worst option for users. Existing installs are stuck on an old (and potentially insecure) version, and new installs are impossible.
Old version, either static or with updates: Bad for users, since they get an old (and potentially insecure) version.
latest/stable: Confusing for users, since the same command installs radically different versions on different machines.
old/stable, consistent default: A bit confusing for users, since snap install foo won’t work any more. But the error message at least lists the channels that do contain something, so the user isn’t completely blocked. Breaks upgrades.
old/stable, inconsistent default: Confusing for users, since the same command installs radically different versions on different machines. Needs store work to support different default tracks for different architectures. Breaks upgrades.
Option 1 (supporting multiple bases for the current version) is the common pattern used outside the snap world by most distributions and ISVs today. It’s the only one that provides a good experience for the user, but for the developer it requires multiple snapcraft.yamls (and sometimes significantly different ones, since snapcraft implicitly changes behaviour for some bases, e.g. switching to incompatible v2 plugins for base: core20).
Option 3.3 (default track differing by architecture) isn’t supported by the store today. It would be possible to implement without changing snapd or the store’s snapd-facing APIs, but it obviously brings additional confusion for users, and more room for developers to make mistakes.
There’s no clear upgrade path for the options that involve a new track, 3.2 and 3.3. While we’ve discussed mechanisms to allow the server to permanently redirect a client such that it follows a different channel in future, it was decided that going against user intent like that was not acceptable. So while we could make default tracks architecture-specific so that the store can tell new installs to follow old/stable, there’s no way to prevent existing installs from getting stuck on latest/stable where the architecture is no longer supported.
The problem of base architecture support doesn’t just affect i386. riscv64 only exists from Ubuntu 20.04 LTS onwards, so snaps that target core, core16 or core18 won’t be able to build for riscv64. This isn’t a big problem for a nascent architecture, but will become one as it matures. We’ll also presumably drop other architectures, most probably armhf to start with, in the coming years.
On epochs: they’re designed to provide multi-step upgrades and aren’t really relevant here. If the new revisions are released only on !i386 architectures, the upgrade would never be considered anyway.
In my particular case, I need iptables 1.8 since it builds nft and legacy backends and I need my snap (ufw) to be able to use one or the other at runtime. iptables 1.8 is not in 18.04 but I really wanted to stage-packages to benefit from security/sru bug fixes since I could rely on Ubuntu quality resulting from millions of users’ use of the official Ubuntu builds for supported architectures rather than my custom build (there were a lot of bugs on the road to a robust iptables-nft so I’m apprehensive of diverging from Ubuntu’s version and patches). I was also concerned that if I built from upstream source that I might run into maintenance issues for the i386 build at some point when upstream might start to not care any more about i386. Of course, I could move to core18 and point to the Ubuntu git branch for iptables for 20.04 and watch for iptables USNs and trigger rebuilds when I see them; while the builds are ‘unofficial’ they would at least come from official Ubuntu sources, which is perhaps a reasonable trade-off*.
Considering your response (thanks!), it sounds like the best path forward is to build on core18 and point to the 20.04 branch of iptables. I then need to decide if I want to support the i386 arch in latest/stable for the duration of 20.04 or to drop it, but that is an easier decision to make.
[*] I’d have to then watch for USNs for iptables and rebuild rather than relying on snap USN notifications. The snap USN notifications service could be adjusted to account for this sort of thing… (ie, using an old base for stage-packages but building from official git Ubuntu sources/similar from a different release than the base).
Well, I forgot to mention that if I go with route ‘1’, that is kicking the can in some ways. When 18.04 falls out of standard support in 3 years, I will be faced with a new decision since I’ll want to move to core20 or core22 to be on a supported base where i386 necessarily is dropped. I probably won’t care about supporting i386 at that time since I only planned to support i386 for the duration of 18.04, but then I’m not sure how to best handle dropping an arch, which brings us to your point about other architectures (I actually brought up this topic to discuss the larger picture and used my i386 case as an example ).
Is the current thinking for dropping an arch documentation, adding a deprecation warning at some point, followed by an obsolete warning in the final revision for the arch, then removing the arch from the track (ie, so existing users don’t get upgrades but there can be no new ones)?
@wgrant - note, while this is probably the right choice for me, I brought up the topic in a generic fashion initially so the discussion could inform best practices for others. It sounds like you are recommending variants of your option ‘1’ as the best practice since other options (with the possible exception of 3.2) lead to confusion?
builds and runs with VA (Optionally, from the xbmc PPA)
i386
builds and runs without VA (From the xbmc PPA)
not supported by core20
armhf
not supported by xbmc PPA
Runs with graphics limitations
arm64
not supported by xbmc PPA
Runs with graphics limitations
I think it will be best to publish core20 based builds to amd64, armhf and arm64, but keep an additional core18 build for i386. This isn’t as simple to automate as I’d like, but provides the best support available for each architecture.
).