Hi folks,
I’ve been working on various graphical snaps for Ubuntu Core, with the aim of creating solutions for digital kiosks and signage.
One issue I have encountered is dealing with GPUs: libGL and friends. Currently all my graphical snaps end up pulling in all of MESA to get GL working. However this is not very flexible:
MESA supports most desktop chips quite well, but its support for embedded devices is far more limited
in the embedded world, most vendors just supply a binary blob of working GPU driver (often a custom kernel to match that) and libGL combination.
This does imply GL drivers should live outside the application snap, but that raises the question: where should they live?
If the device requires a custom kernel, that will require a custom core snap for that device - so is it best for the GL drivers to be included in the core snap? Or a separate gl-drivers snap, somehow tied closely to that particular core, and using content interface to share the userspace drivers to the application snap?
I’m curious if there is any established thought on this problem.
Thanks
-G
i would actually expect the display-server snap to ship the blobs along, perhaps mapped via a layout (i.e. raspberry pi by default ships them in /opt/vc/lib and if you try to re-use existing (non-ubuntu) application binaries in a snap they will likely be linked against that dir)
note that we do not ever allow cusom core snaps in UbuntuCore (thats not supportable), so the libs would have to come via some other mechanism.
That is a decent idea. Though I’m not a fan of tying a display server snap (should be hardware agnostic) to hardware. We’d end up with mir-kiosk-mesa, mir-kiosk-nvidia-390, mir-kiosk-mali-something-for-some-board-rev… And installing the wrong mir-kiosk-* for the core running, it just won’t work (and no dependency system to prevent it).
But on the other hand, for a device vendor, they can tie the display server snap with the core easily, and test graphics easily in that situation. So I can see the merit of that approach.
A problem remains with how we share those GL libs with an application snap. Content interface is an obvious choice, but then need a standard interface name to be shared between display-server and application snap. I fear if it’s not standard, we’ll end up with only very specific combinations of snaps ever working together.
Pie-in-the-sky thinking, I’d like the opengl interface to be one that a third-party snap can be the slot of, in such a way that any snap that plugs into that slot, has the gl libs bind-mounted into its snap namespace.
Would mean gl libs live in their own snaps, we’re using a pre-existing interface, and application snaps don’t need to do anything special. But I know well that ideas are easy…
-G
hmm, not really, you’d simply have the mir-server armhf snap ship all possible GL libs (mali, vc4, pvr, whatever) and have some hook to pick the right ones based on the detected HW at install time … x86 would have nvidia, intel and amd (i think it has them anyway already from the mesa dri packages IIRC). the snap would get slightly bigger but i dont think this would be a significant lot.
Hmm but then the responsibility for shipping working gl drivers for any chipset will rest with the Mir team. There’s no way we can test all the possible hardware options. And I’m not sure we’d have legal rights to distribute the blobs either.
Hence me thinking it cleaner that device vendors take responsibility for whatever snap ships their device’s GL driver.
That would then be either gadget or kernel snap which would need to be extended in the right way to allow shipping these libs (driver modules themselves would be in the kernel snap anyway) and bind mount them to the right place so the GL interface exposes them to clients… today the kernel snap does not allow anything but /lib/modules and /lib/firmware though.
I guess the other obvious question is what this means for the AppArmor policies. The existing opengl interface covers the requirements of DRI/Mesa, Nvidia, and some others by the look of it.
If you’ve got a random vendor libGL talking to a matching kernel driver, it seems likely that these won’t be sufficient. Presumably we don’t want a custom snapd for each of these embedded devices?
What i was proposing (via kernel snap shipment) would ship driver and libGLES2/GL in the same kernel snap, they can talk to each other without special treatment in that case (in the UbuntuCore rootfs the libs would have to be placed where the driver expects to find them) … to expose them to client snaps (mir-kiosk, application snaps etc) the libs would have to be mapped to /var/lib/snapd/lib/gl* where the opengl interface can just pick them up and map them to the right places in /var/lib/snapd/hostfs/ in the application snap environment.
We’re talking about a library here though: the code contained in it will be running in the context of an application (or display server), which will be in a different snap than the kernel. So I don’t think you can ignore the interface between the vendor libGL and kernel.
indeed you would have to limit the allowed setups to DRI, but thats doable i think (mali definitely uses DRI, raspberry PI vc4 is usable via DRI, newer PVR and vivante should also use DRI). we can surely also extend the opengl interface to more exotic setups if needed. but simply saying you need DRI/KMS support with your blobs should be a good start to cover a wide range.
What we need though is collaboration and commitment from the core team since we’d first need changes to the kernel snap design to allow more than just modules and firmware to be shipped.
that actually reflected plain android setups that we hooked in to.
if you want an UbuntuCore kiosk install that runs on an android-only system (where you only have an original android kernel patched for snapd) you might indeed need such specific hacks, but i think this would be the exception. our focus should be on plain linux setups here for the start (we can probably later have an android-gles inteface for such special cases that can then grow in multiple directions)
This will rule out any proprietary BSP based graphics drivers on Ubuntu Core that have to be built directly along with the kernel (mali, vivante, SGX/pvr etc) which enforces a hard dependency between kernel and drivers/libs that snaps can not reflect. What was the reason to not simply allow kernel snaps to ship the proprietary bits in /usr/lib (along with the already allowed /lib/modules and /lib/firmware) and simply expose them via /usr/lib/gl (or whatever the opengl interface offers today as generic place) ?
Well, i’m not sure how much we care about a PC scenario where you likely have the libs and modules shipped by the underlying classic system anyway.
It is definitely a bit weird to separate the drivers and libs from the kernel in the proprietary Core BSP scenario unless we only want to support free and reverse engineered drivers (like lima, evnativ etc with very limited features though) … my guess is that in the digital-signage ARM case the proprietary use-case is bigger though.
I agree that the two scenarios perhaps require two implementations to support both use-cases.
How will this affect the issue I raised above where we bundle a large number of chunky libraries in each desktop snap? Having numerous copies of all the GPU drivers, plus all their dependencies is a space hog. We really need to fix that.
