Model assertions for core18

snapd
1

Hi,

some of the snapd team had a discussion about the model assertions for core18. After some deliberation and tweaking we decided to create the following model assertions:

ubuntu-core-18-amd64

{
    "type": "model",
    "authority-id": "canonical",
    "brand-id": "canonical",
    "series": "16",
    "model": "ubuntu-core-18-amd64",
    "display-name":"Ubuntu Core 18 (amd64)",
    "architecture": "amd64",
    "base": "core18",
    "gadget": "pc=18",
    "kernel": "pc-kernel=18",
}

Compared to the previous draft of the model assertion the “display-name” change to include the architecture and the gadget includes a channel now.

ubuntu-core-18-i386

{
    "type": "model",
    "authority-id": "canonical",
    "brand-id": "canonical",
    "series": "16",
    "model": "ubuntu-core-18-i386",
    "display-name":"Ubuntu Core 18 (i386)",
    "architecture": "i386",
    "base": "core18",
    "gadget": "pc=18",
    "kernel": "pc-kernel=18",
}

This has the corresponding changes to the amd64 version above.

ubuntu-core-18-pi3

{
    "type": "model",
    "authority-id": "canonical",
    "brand-id": "canonical",
    "series": "16",
    "model": "ubuntu-core-18-pi3",
    "display-name":"Ubuntu Core 18 (pi3)",
    "architecture": "armhf",
    "base": "core18",
    "gadget": "pi3=18",
    "kernel": "pi-kernel=18-pi3",
}

Compared to the previous draft this pulls the gadget from the 18 channel, adds “pi3” to the displayname and changes the kernel name to something more generic that is then pulled from a pi3 specific kernel track.

ubuntu-core-18-pi2

{
    "type": "model",
    "authority-id": "canonical",
    "brand-id": "canonical",
    "series": "16",
    "model": "ubuntu-core-18-pi2",
    "display-name":"Ubuntu Core 18 (pi2)",
    "architecture": "armhf",
    "base": "core18",
    "gadget": "pi2=18",
    "kernel": "pi-kernel=18-pi2",
}

This has the corresponding changes to the pi3 version above.

ubuntu-core-18-cm3

{
    "type": "model",
    "authority-id": "canonical",
    "brand-id": "canonical",
    "series": "16",
    "model": "core-cm3-18",
    "display-name":"Ubuntu Core 18 (cm3)",
    "architecture": "armhf",
    "base": "core18",
    "gadget": "cm3=18",
    "kernel": "pi-kernel=18-cm3",
}

This has the corresponding changes to the pi3 version above.

ubuntu-core-18-dragonboard

{
    "type": "model",
    "authority-id": "canonical",
    "brand-id": "canonical",
    "series": "16",
    "model": "core-dragonboard-18",
    "display-name":"Ubuntu Core 18 (dragonboard)",
    "architecture": "arm64",
    "base": "core18",
    "gadget": "dragonboard=18",
    "kernel": "dragonboard-kernel=18",
}

Compared to the previous draft this pulls the gadget from the 18 channel and adds “dragonboard” to the display-name.

[edit: fixed cm3 omissions]

2 Likes
2

Did you mean to use “Ubuntu Core 18” there? To be consistent with other models it ought to say “Ubuntu Core 18 (cm3)”

Why is cm3 not using release specific track like pi2 does?

3

what happened to

i had high hopes we’d have a single universal pi image (and a single universal gadget) in core 18 …

2 Likes
4

Thanks @zyga-snapd! Those were indeed mistakes, I corrected them now.

1 Like
5

Hey @ogra. The single universal pi image has some appeal. I forked with it and updated it to be a replacement for the existing gadgets in https://github.com/mvo5/pi-gadget and it looks nice.

However when we discussed this internally one concern that came up was that the different PI models are different and have differences in chipset, IO ports etc. As gadgets become smarter having only a single gadget for all PIs may limit what we can expose via the gadget. Therefore the decision was taken to keep them separate.

6

The chipsets are close enough for upstream to support a single bootloader for all boards (even the pi1 and zero’s), in fact there are only two SoC chipsets supported by core (pi2 and the whole rest)).

It was us that introduced an artificial separation here. Would we have been able to go without u-boot, we’d likely have gone with a single unified image from the start simply using upstreams bootloader and successively have added boards as they appeared, after all we have a unified kernel as well, doing exactly the same.

While it is true that there are some differences in additional IO ports, there are no conflicting ones at all (and will never be since hardware sockets for addon boards (the Pi HATs) have to stay compatible between the boards). As long as the gadget.yaml covers all of the interfaces there is no problem (if you would want to distinguish between the boards your snap can do that easily, all snaps have read access to /proc/device-tree/model where the exact board name and revision is provided). As gadgets get smarter I would actually expect them to hide the superfluous interfaces and not become more limiting in any way (that does not really seem like “smarter” TBH).

Regarding the “they did a change that allows having a single image… tomorrow they can do a change that requires different images again” (quoting from IRC) that gustavo brought up as the main reason to turn this down: i do not see why “they” would do this, the feature we are using has been upstream since two years and is widely used in upstreams images, we just did not make any use of it before and upstream would get into the same trouble when they removed it.

Also the point of transitioning from or to unified gadgets he made during this discussion is a bit moot given that we can not drop the core 16 setup and will have to support it due to the fact that we allow core 16->18 upgrades, so the new unified snap would only be usable for new installs of new images anyway here, but it would be a future path out of that per-board limitation.

All of that said, it is not in my power to make this decision and i have to (and will) respect what you guys decided here.

I know that we have a few customers waiting for such a feature though and @ondra just asked me if we could at least have the snap as an official canonical signed snap in the store (and I’d appreciate if ondra and i could also have store collaboration rights to it) so we are able to build demo images for said customers without having to use third party signatures.

Edit: it was just also brought to my attention that we’ll soon have 4 Pis (3B+ (same chipset as Pi3 and cm3 but new network card and wlan peripherials)) and soon after there might be a 5th (Pi4 ?) to support, that will quickly get unmaintainable for us…

7

Why is the series always 16 (not 18)?

8

series in that context indicates a collection of evolving but backward compatible assertion formats and also snap namespace, core 18 is compatible with the core 16 world, all the snap namespace will carry forward

9

and yes, it’s a slightly unfortunate term, it was picked thinking of other uses of series that maybe it would match, but before we had a full idea of how we would evolve assertions and the snap namespace

10

One is a Cortex A7, while the other is a Cortex A53. This is a relevant physical difference instead of artificial, and until we have someone that highlights those differences in a more respectful way and has a good understanding of how to keep that zoo under control if it goes bad, I’d recommend against unifying the images.

11

We currently only provide armhf images, so only using the cortex-a7 mode in all Pi images regardless if the SoC is A53 or A7.

Once there will be arm64 (A53) based images we’d indeed only be able to cover the 64bit side with them, but these would be a completely new set of images as well as a completely new set of potential gadget snaps (these could indeed as well be using a unified A53 only gadget but thats for a different discussion).

(also note that upstream calls the 64bit (A53) mode of the Pi3 family “unsupported” yet they offer firmware for it, they do not actually fully support images using it)

12

The images had to be different for them to work until very recently, because those are very different chips. The fact things are now fixed so they can work doesn’t change the fact that those are still very different chips, even though we never provided 64bit images. If we ship a single image, we’ll never be able to update the kernel of a deployed system independently between the two different boards.

Producing new images is irrelevant here. It’s the images we’re building that I’m worried about.

13

The images were different because we thought we had no easy way (because before nobody researched the feature the unified gadget now uses) to load different uboot.bin’s, which is the only thing that the current images actually differ in.

They all use the same first stage bootloader firmware from upstream and use the same kernel snap. During boot the firmware chain-loads uboot to then do our Ubuntu Core scriptery required by snapd (to support roll-back of core and kernel). Uboot then loads kernel and core and boots.

The upstream feature the unified gadget uses (which is used and supported upstream since at least 2016, it is nothing new at all) now allows to switch to the right uboot.bin matching the hardware during the initialization of the upstream firmware. It checks the ROM and loads the correct uboot.bin for the specific Pi it runs on. So it unifies this one piece that always forced us into separate gadgets and thus into separate images. There has never been another reason to have separate Pi images beyond the different uboot.bin’s.

I dont get why updating the pi2-kernel snap is any different between single or multiple images here, the kernel supports all boards equally today and will not stop to do so, no matter if there is a single gadget or multiple gadgets. Likewise the firmware supports all boards and will load the correct dtb’s and uboot binaries for them, completely independent from the fact if there are separate gadgets or one unified one.

Me too, which is why i made this proposal in the first place. Today you support 3 (soon to be 4) separate pi images for core 16 for 5 years (to my knowledge we allow people to refrain from upgrading to 18 which means we need to keep supporting them til 2021)… going forward you will have to support another set of three (or 4) for core 18, plus an upgrade path.

So we are at least at 6 images, adding 3 more (likely more than three over time as new Pi hardware shows up) with each release.

My proposal allows to only ever add one new image with each new core release and it allows even to extend this one image to support new upcoming Pi hardware on the fly without forcing a new image into the set, by simply adding the new u-boot as a part to the snapcraft.yaml of the gadget.

What i’m proposing is to allow new core18 installs to only go via the new unified image here while keeping the core16 setup alive for the 5 years that core16 is supported, plus an upgrade path for them, keeping the old separated design for the upgraded images in the core18 cycle but not beyond.

So we dont have to deal with an infinitely growing amount of images eventually.

14

Right, that’s the right question. If everything was the same there would be no reason for different images. But everything shouldn’t be the same otherwise we cannot distinguish the completely different devices.

So with that mindset, the different images need to at least point to different snaps or different channels for the kernel snap so that they can be updated independently if we have to. They will also have a different model so that we can make snapd treat them differently if we need to.

15

Right, i get what you are after here, what i do not understand is why we would have to or need to do this (perhaps i’m missing a piece of the picture, but i would like to understand which one if i do).

All gadgets today ship the same set of all dtbs for all devices already, due to that they are universal on that level already … at the same time all gadgets ship the same upstream bootloader firmware and have to (because of the dtb’s) use the same kernel snap which in turn is already universally supporting all existing boards.

Having different models today is caused by the fact that we have different gadgets, which we do because it was not possible to have a unified u-boot binary for all of the boards (and thus no unified gadget). The dtb’s in all the gadgets are tieing them to a certain major kernel version (you can not boot a 4.4 kernel with 4.15 dtbs or vice versa).

If a change in kernel or dtb’s causes an update today, all gadgets have to be updated alongside due to the above design limitation of shared dtb’s.

What else would trigger a separate update of the gadgets apart from fixing a u-boot bug or making a change in config.txt (both of which the unified gadget supports on a per-board base already) ?

16

Just my 2 cents here.

I’d propose for the moment to ignore fact that dtbs are shipped inside gadget (making hard dependency between kernel and gadget). This really should be job for ubuntu-image to learn how to lift dtbs from kernel snap when building image. Unless I’m wrong and there is then point when those dtbs are read again from gadget at run time?
I hope this is something we can address when Core 18 is out or may be as part of the process? Clean the house a bit. That can probably even make gadgets compatible between Core 16 and 18
Anyway this was a bit off topic

Now to the topic here, leaving whole dtb angle out.
From what I could see in development from Raspberry foundation, they are actively working on features allowing universal images. For example kernel command line has keywords and bootloader replaces them with right values. Like tty device matching actual hw. Same mechanism is there to pick right kernel in our case u-boot. It does not seem like experiment from their side, but more like a commitment to be able produce single image. They even support Pi nano and other hw from same image.

Which in some way makes lot of sense to end users/developers.
I have on my desk mix of Pi2, Pi3, Pi3+ and frankly they look quite alike. So being able to download one image and boot is very good experience. And it is experience provided by many others alternative OS flavours. So I’d vote in favour to match this experience.

Especially if all we need is to load correct u-boot as rest of the gadget snap is identical.

Or question from another angle.
Do we have separate image for Pi3+? As far as I can see we plant to support it from Pi3 image.
It has different wifi chip, it has different ethernet chip. Why same image as for Pi3 then?

17

Images for amd64/i386 based on the above model assertions are now available at http://people.canonical.com/~mvo/core18/ for testing. We will continue to work on dragonboard and PI images next.

18

Is there already a possibility to upgrade a pi3 with core16 to core18 just using snap?