The only workaround that doesn’t require “all” snaps to be rebuilt is for snapd to not mount the drivers/export stuff in SNAP_LIBRARY_PATH unless the host userspace matches the snap base.
Maybe the option is to stop symlinking
/var/lib/snapd/lib/gl. It’s not part of the Nvidia drivers, and a version provided by the snap should be able to load the Nvidia libraries (provided those binary blobs are in fact compatible).
The main questions in making that change would be:
- Are there supported systems out there not using glvnd but providing proprietary Nvidia drivers that snapd actually uses?
- Are there supported EGL snaps out there that are not shipping (or plugging a content snap that ships) glvnd to access the Mesa drivers?
I recall that in pre glvnd days there could have been libEGL.so.1 shipped by nvidia drivers. I was not able to locate exact reference in the drivers release notes but found this: https://forums.developer.nvidia.com/t/multiple-glx-client-libraries-in-the-nvidia-linux-driver-installer-package/41308 which discusses a glvnd and non-glvnd installs when 361 version of drivers was released, so back in 2016.
Just checked that the nvidia -384 driver on 16.04 ships libEGL:
/usr/lib/nvidia-384/libEGL.so /usr/lib/nvidia-384/libEGL.so.1 /usr/lib/nvidia-384/libEGL_nvidia.so.0
It looks like the libEGL.so.1 in that package is a libGLdispatch (the predecessor to glvnd) based driver seelctor. It’s likely the
libEGL_nvidia.so.0 file could be loaded by glvnd, but there’s no
egl_vendor.d json metadata file to tell it that the driver exists.
In the other direction, a snap built with
base: core will probably not be using the glvnd version of
libEGL.so.1, so would not be able to use the
egl_vendor.d metadata provided by a newer host system’s Nvidia drivers. But with that said it can also fail to use that host system’s
libEGL.so.1 as in the Impish case that started this thread.
Just got this issue after updating Telegram to the new 3.2.0 release on Ubuntu 21.10:
$ telegram-desktop telegram-desktop: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /var/lib/snapd/lib/gl/libEGL.so.1)
$ snap --version snap 2.53+21.10ubuntu1 snapd 2.53+21.10ubuntu1 series 16 ubuntu 21.10 kernel 5.13.0-20-generic
Mesa Intel® HD Graphics 630 (KBL GT2) + NVIDIA 930MX (Ubuntu’s 470 driver) in on-demand mode.
So if I’m reading this right, in either scenario libEGL_nvidia would/could not be used and there’s either no metadata ti look it up, or the metadata format is different?
How about vulkan? We also try to pull in the vulkan ICD files in snap-confine.
libEGL_nvidia.so.0 is a proprietary binary provided by Nvidia, and is built on whatever system Nvidia considers to be their lowest common denominator. I don’t think it is currently a problem for any of the base snaps we support.
libEGL.so.1 in current Ubuntu releases is an open source driver multiplexer (part of the libglvnd source package). It is built against whatever libraries are current in the corresponding distro release. In 21.10, this results in a dependency on the
GLIBC_2.33 symbol version.
I think the right way to handle things would be to use the copy of the driver multiplexer provided by the snap (e.g. by the gnome platform snap), which should be able to load the Nvidia driver even on modern systems.
The problem is legacy pre-glvnd systems like Ubuntu 16.04, which use a different Nvidia-only dispatch library (GLdispatch) and don’t ship the json metadata needed for glvnd to function. In that case, we really do want the
libEGL.so.1 from the host system.
Are you aware of any other indicators other that lack of nvidia files under egl_vendor.d that would suggest that we need libEGL from host? I can tweak s-c to avoid symlinking in such case. This would need to be time aligned with gnome platform snap shipping the right libraries too.
The other combination that could cause trouble is a snap that isn’t built to use glvnd (which probably just means some old core16 snaps). But I’m not sure whether they would function at all on an Ubuntu 21.10/Nvidia system.
I posted some test instructions to see if we can improve behaviour for Firefox here by preferencing the snap’s driver multiplexer here:
If you’ve got a system where you can reproduce, perhaps that might help determine whether the change will work.
@Trevinho I’ve just noticed that the Telegram snap does start on 21.10 due to this issue.
telegram-desktop: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /var/lib/snapd/lib/gl/libEGL.so.1)
Can confirm. And I can also confirm that @jamesh’s fix works here. This or something similar needs to get implemented soon. The UX for the average desktop user is quite terrible.
Just a quick update on this, @jamesh is working on a solution for this in snapd. More to come on this soon!
I’ve put together a branch that attempts to fix things from the snapd side here:
I don’t have any Nvidia hardware to try it myself, but it should have a similar effect to the LD_LIBRARY_PATH hack I asked people to try earlier.
The CI should produce a test build of the snapd snap that people can test out. I’ll post more details about how people can test it when it is available.
Thanks. This looks promising. I think the snapd snap produced by the CI run is only accessible if one is logged in to their github account. Would it make sense to provide grab that snap and provide it at people.canonical.com leaving a link in the topic?
@jamesh that looks like a sensible approach: I hope we get confirmation that it works in practice.
I still wonder if we need, in principle, to hack
__EGL_VENDOR_LIBRARY_DIRS (but that probably only enables hybrid setups that were already broken before 21.10)
Both the Snapcraft desktop extensions and the old snapcraft-desktop-helpers scripts already set
__EGL_VENDOR_LIBRARY_DIRS. For example:
As well as enabling the Nvidia drivers, this is necessary to load the Mesa backend provided by e.g. the Gnome platform snap.
Oh right, I keep forgetting about those. (As, unfortunately, I don’t find them useful for IoT snaps.)
I’ve curated a list of snaps without a base that plug opengl, we should ensure some sampling of these are tested for regressions.
adrift agent ags alarabiya-desktop allow2automate alphaapp amoveo-wallet ampareimagetopdf amparepdftoimage amparepngtoico amparewakeonlan anxing ao asarui atomify audovia auryo autoskola-free6 balls2 baugeschichte bayam bitshares2-light blender-tpaw bottle buka burdirc bussard buzyteam bzflag bzflag-tjhanson caprice32 cass castersoundboard cclite checksum-validator chinese-cal clari3d-lite-64 codebreakers codenamelt college-mastodon collision colmap-mardy crrcsim-simulator cumulonimbus deepin-image-viewer deepin-music deepin-voice-recorder demo-gtk-common-themes desktop-habitica devrantron dino djv douban-fm drmips electronic-wechat electron-quick-start electrontestapp eloleo elrond eos-voter epipolar-consistency e-tools eureka-doom-editor euruspro-desktop evernote-web-client exers extia-webapp facebook-webapp-mardy facebook-webapp fcole90-hexgl-webapp fiwe flacon-tabetai flare-rpg foobillard-plus freeorion-agrrr3 fromscratch fugio functy git-cola goldendictionary google-webapp goxel granatier graphics-debug-tools-bboozzoo grumpy heb12-desktop hello-world-electron hexexplorer-snap hiri hmi-gspe huggle icq-im idid imagecomparer imagenes imaginary-teleprompter instagraph jimbodicomviewer kanagame kiosceditor kiosc konstructs-client koombo k-sudoku langly lanto lattam librealsense-chenhan librealsense love2d lxi-tools lyricpad mcomix-tabetai meshlab-mardy minetest-luk3yx-3 mitk mmex molden monento moonplayer morpion movie-monad mrxecplayer musicarley musixmatch mve-mardy mve myposoid-windows myteam nanowallet neuronify nocturn-mardy nordvpn-electron notepadqq numnom offs onlykey-app openhantek orangecalc paintsupreme-3d panorama parity-ui passwordlocker pathplanner pencilsheep pin-town planet-jumper planetlander plexmediaserver pocketmine-server-manager pockit posecalib postman prboom-plus-beidl programmer protogrid proton-wallet pyside2-hello-world qcheckers qr-code-generator-desktop qstamina-snap quakespasm-beidl quelea realsense-samples reicast relay remote rfid-app riminder rockscissorspaperlizardspock-snap sagudev-aranym savagexr-seaeyeaya scrumpy seekwell sftpclient shipgunner siilihai-client simplescreenrecorder sirrujak-sciencefair sit-lpc-app skrifa-lite skrifa slashlock smallpasskeep snapd-hacker-toolbelt solitude sololearn soracom-console special-delivery spelunky squarehead ss-qt starruler2 start-and-doc stifts-terminal stochsd-electron stonscipap-snap submix supercalc-snap sword symgrpmad tbinge test872334 test-snapd-glxgears theia-mardy thunderdocs todo-antrax torgo transtracdevice tusk twistypuzzle typora-alanzanattadev ubuntu-mate-launchpad udemy unofficial-zalo vessel viber-unofficial viojh-snap10 virtualjaguar-jz visualsfm-mardy volleyball2d voxelands-luk3yx vtest13 vtest-chrome webdingding webengine-app wmx-kiosk-session wonder-reader workchat wps-office-all-lang-no-internet wps-office-multilang wps-office wuziqi wyzepal xbts-light yamagi-quake2-beidl yd youdaonote
The CI job has finished, so there is a build available here as an artifact:
If you are logged in to Github, it should be possible to download the “snap-files” artifact at the bottom of that page. You can install the modified snapd with the following commands:
unzip snap-files.zip sudo snap install --dangerous snapd_*.snap
After you’ve finished testing, switch back to the stable release from the store by running the following:
sudo snap refresh --amend --stable snapd
If you’ve got a system with an Nvidia GPU, I’d appreciate feedback on how this build runs. Some things that would be useful to test:
If you’re running Ubuntu 21.10, does this fix any snaps you use that are broken with stable snapd? Before testing the snap, run the following command:
sudo /usr/lib/snapd/snap-discard-ns $snap_name`
This will ensure you’re not using a sandbox set up by the old snapd version.
If you’re running an older version of Ubuntu, it would also be useful to test that this doesn’t cause regressions for apps you use.
base: coresnaps work? @kenvandine’s post above contains a list, but it’s probably best to pick snaps that appear to be somewhat actively maintained. Some look abandoned or uploaded as tests.
If you find snaps that malfunction with this snapd build, it would also be useful to know whether they function correctly with stable snapd.
Reports of both successes and failures are appreciated.
telegram-desktop started working again for me with an update in the last few days. It now uses
core20, so that’s where that’s coming from.
maxi@maxi-desktop:~$ snap install flokk-contacts flokk-contacts 1.0.1 from gskinner (gskinner-apps) installed maxi@maxi-desktop:~$ snap run flokk-contacts /snap/flokk-contacts/11/flokk-contacts: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /var/lib/snapd/lib/gl/libEGL.so.1) maxi@maxi-desktop:~$ sudo snap install --dangerous ~/Downloads/snapd_2.53.1+gitafcf491.afcf491-dirty_amd64.snap [sudo] password for maxi: 2021-11-08T16:17:49+01:00 INFO Waiting for automatic snapd restart... snapd 2.53.1+gitafcf491.afcf491-dirty installed maxi@maxi-desktop:~$ sudo /usr/lib/snapd/snap-discard-ns flokk-contacts maxi@maxi-desktop:~$ snap run flokk-contacts [...]
flokk-contacts was broken with stable snapd, but launches again with this fix : )