Nvidia drivers in a MIR / Ubuntu frame snap

jogi · June 7, 2022, 7:31pm

Hi,

I found several guides and questions related to my issue but not exactly matching it.

ok I’m quite new in the world of snaps, snapcraft and especially MIR / Ubuntu frame. All the input is a bit overwhelming and sometimes confusing due to the move (?) from MIR to Ubuntu frame.

Anyways. What I am trying to achieve is creating and using a secure kiosk based on Ubuntu frame. For that I have used the following repo as my snapcraft base: https://github.com/ogra1/electron-kiosk-wayland

it works so far but now I encounter an issue on a new device which is different:

the OS where the snap is installed to is Ubuntu 22.04
the snap itself uses core20 (which might be a prob already due to the KMS + nvidia thingy!?)
the system has proprietary nvidia drivers installed
I made no changes in the snapcraft.yml regarding nvidia (bc i have no idea what and where)

IIRC the problems were not there up to the point where I installed the proprietary nvidia drivers but I cannot verify that atm unfortunately (the system is not nearby anymore).

I found this: https://<remove-this>discourse<DOT>ubuntu<DOT>com/t/the-graphics-core20-snap-interface/23000 but still not sure how exactly I need to configure it to fit my snap or if that is even related or would solve my issues or not…

Also I found this: https://<remove-this>github<DOT>com/MirServer/ubuntu-frame/pull/46/files#diff-56759910381a014fecfd7556dd72ddd68c747d922a5b7df2044b9ce7c552f5f5R109-R112 maybe that is needed??

Then I found this: https://github.com/MirServer/mir/issues/1359 by @alan_g but not sure if that is needed inside the snap or outside or both?! or needed at all?!

sorry but I am so confused and no idea where my issues come from and how to fix…

the log shows an issue regarding KMS (as said I haven’t done any action in that direction yet as I dont know even where to start):

→ 2022-06-07T15:54:42Z <....> gbm-kms: KMS not supported (Couldn't get DRM resources: Operation not supported)

that might be the reason?

full log (snap logs ubuntu-frame -n 300)

2022-06-07T15:54:38Z systemd[1]: Started Service for snap application ubuntu-frame.daemon.
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: + exec nohup /snap/ubuntu-frame/1902/bin/run-frame /snap/ubuntu-frame/1902/usr/local/bin/frame
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: + [ /snap/ubuntu-frame/1902/usr/local/bin/frame = --help ]
2022-06-07T15:54:39Z ubuntu-frame.daemon[2663]: + dirname /run/user/0/snap.ubuntu-frame
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: + XDG_RUNTIME_DIR=/run/user/0
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: + export XDG_RUNTIME_DIR
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: + mkdir -p /run/user/0 -m 700
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: + [ -n  ]
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: + exec /snap/ubuntu-frame/1902/usr/local/bin/frame
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.818423] <information> mirserver: Starting
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.820937] < - debug - > mirserver: Using Linux VT subsystem for session management
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.821019] <information> mircommon: Loading modules from: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.821037] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/graphics-gbm-kms.so.19
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.821042] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/server-x11.so.19
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.821045] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/input-evdev.so.8
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.822807] < - debug - > gbm-kms: Quirks: checking device with devnode: /dev/dri/card1, driver nvidia
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.823076] < - debug - > gbm-kms: MIR_MESA_KMS_DISABLE_MODESET_PROBE is set
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.823081] < -warning- > gbm-kms: Failed to detect whether device /dev/dri/card1 supports KMS, continuing with lower confidence
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.823108] <information> mirserver: Found graphics driver: mir:gbm-kms (version 2.7.0) Support priority: 128
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.824163] <information> mirserver: Found graphics driver: mir:x11 (version 2.7.0) Support priority: 0
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.824363] <information> mirserver: Selected driver: mir:gbm-kms (version 2.7.0)
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.824536] < - debug - > gbm-kms: Quirks: checking device with devnode: /dev/dri/card1, driver nvidia
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.824575] < - debug - > gbm-kms: MIR_MESA_KMS_DISABLE_MODESET_PROBE is set
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.824578] < -warning- > gbm-kms: Failed to detect whether device /dev/dri/card1 supports KMS, but continuing anyway
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.824585] <information> gbm-kms: Using DRM device /dev/dri/card1
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.824617] < - debug - > gbm-kms: Quirks: checking device with devnode: /dev/dri/card0, driver i915
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.824662] <information> gbm-kms: Using DRM device /dev/dri/card0
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.942331] <information> mircommon: Loading modules from: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.942368] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/graphics-gbm-kms.so.19
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.942378] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/server-x11.so.19
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.942381] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/input-evdev.so.8
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.943192] < - debug - > gbm-kms: Quirks: checking device with devnode: /dev/dri/card1, driver nvidia
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.971648] <information> gbm-kms: Detected software renderer: llvmpipe (LLVM 12.0.0, 256 bits)
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.972370] < - debug - > gbm-kms: Quirks: checking device with devnode: /dev/dri/card0, driver i915
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.983493] <information> mirserver: Found graphics driver: mir:gbm-kms (version 2.7.0) Support priority: 256
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.983510] <information> mirserver: Found graphics driver: mir:x11 (version 2.7.0) Support priority: 0
2022-06-07T15:54:39Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:39.983545] <information> mirserver: Selected driver: mir:gbm-kms (version 2.7.0)
2022-06-07T15:54:42Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:42.708018] <information> gbm-kms: DRM device details:
2022-06-07T15:54:42Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:42.708058] <information> gbm-kms: /dev/dri/card1: using driver nvidia-drm [NVIDIA DRM driver] (version: 0.0.0 driver date: 20160202)
2022-06-07T15:54:42Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:42.708091] <information> gbm-kms: 	KMS not supported (Couldn't get DRM resources: Operation not supported)
2022-06-07T15:54:42Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:42.708099] <information> gbm-kms: /dev/dri/card0: using driver i915 [Intel Graphics] (version: 1.6.0 driver date: 20201103)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.043179] <information> gbm-kms: 	Output: DisplayPort-1 (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047194] <information> gbm-kms: 	Output: HDMI-A-1 (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047236] <information> gbm-kms: 	Output: DisplayPort-2 (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047251] <information> gbm-kms: 	Output: HDMI-A-2 (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047268] <information> gbm-kms: 	Output: DisplayPort-3 (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047285] <information> gbm-kms: 	Output: HDMI-A-3 (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047302] <information> gbm-kms: 	Output: DisplayPort-4 (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047318] <information> gbm-kms: 	Output: HDMI-A-4 (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047325] < - debug - > miral: Display config using layout: 'default'
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:44.047350] <information> miral: Display config:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: 8>< ---------------------------------------------------
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: layouts:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: # keys here are layout labels (used for atomically switching between them)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: # when enabling displays, surfaces should be matched in reverse recency order
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:   default:                         # the default layout
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:     cards:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:     # a list of cards (currently matched by card-id)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:     - card-id: 0
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:       DisplayPort-1:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:         # (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:       HDMI-A-1:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:         # (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:       DisplayPort-2:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:         # (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:       HDMI-A-2:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:         # (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:       DisplayPort-3:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:         # (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:       HDMI-A-3:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:         # (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:       DisplayPort-4:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:         # (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:       HDMI-A-4:
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]:         # (disconnected)
2022-06-07T15:54:44Z ubuntu-frame.daemon[1736]: 8>< ---------------------------------------------------
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: xkbcommon: ERROR: couldn't find a Compose file for locale "en_US.UTF-8"
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.416013] <information> mirserver: Using hardware cursor
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.416836] < -warning- > gbm-kms-buffer-allocator: Failed to bind EGL Display to Wayland display, falling back to software buffers: /build/mir-2t3qc5/mir-2.7.0/src/platform/graphics/egl_extensions.cpp(89): Throw in function mir::graphics::EGLExtensions::WaylandExtensions::WaylandExtensions(EGLDisplay)
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: Dynamic exception type: boost::wrapexcept<std::runtime_error>
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: std::exception::what: EGL display doesn't support EGL_WL_bind_wayland_display
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.416876] <information> gbm-kms-buffer-allocator: Enabled linux-dmabuf import support
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.416984] <information> mircommon: Loading modules from: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.417002] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/graphics-gbm-kms.so.19
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.417015] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/server-x11.so.19
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.417019] <information> mircommon: Loading module: /snap/ubuntu-frame/1902/usr/lib/x86_64-linux-gnu/mir/server-platform/input-evdev.so.8
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.461580] <information> mirserver: Selected input driver: mir:evdev-input (version: 2.7.0)
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.461675] <information> mirserver: Mir version 2.7.0
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466065] <information> mirserver: Initial display configuration:
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466091] <information> mirserver: * Output 1: DisplayPort disconnected
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466093] <information> mirserver: * Output 2: HDMI-A disconnected
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466095] <information> mirserver: * Output 3: DisplayPort disconnected
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466096] <information> mirserver: * Output 4: HDMI-A disconnected
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466098] <information> mirserver: * Output 5: DisplayPort disconnected
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466099] <information> mirserver: * Output 6: HDMI-A disconnected
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466101] <information> mirserver: * Output 7: DisplayPort disconnected
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466102] <information> mirserver: * Output 8: HDMI-A disconnected
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466347] < - debug - > frame: aa_getpeercon() failed for process 1736: Protocol not available
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466362] < - debug - > frame: aa_getpeercon() failed for process 1736: Protocol not available
2022-06-07T15:54:45Z ubuntu-frame.daemon[1736]: [2022-06-07 15:54:45.466366] < - debug - > frame: aa_getpeercon() failed for process 1736: Protocol not available

any help highly appreciated

PS: Sorry for these remove-this tags… i am a new user so not allowed to post more then 2 links in a post

alan_g · June 7, 2022, 11:25pm

There are several points to address here:

Ubuntu Frame uses Mir, it isn’t a replacement or “move”
Mir can run on Nvidia’s eglstream API but that isn’t currently possible with Frame as the snap doesn’t include this support
The graphics-core20 interface allows Mesa (or alternative) gbm-kms implementations to be used (not eglstream)
Nvidia are working on gbm-kms support

If you switch to Mesa drivers, Frame should work.

jogi · June 8, 2022, 3:57am

Ok cool thanks for the clarifications!

Mir can run on Nvidia’s eglstream API but that isn’t currently possible with Frame as the snap doesn’t include this support

Does that mean I could enable eglstream (whatever that is lol) in that snap somehow? If so how? I already build it so i could change whats needed but what?! Sorry I’m not that graphic aware and just started with snapcraft. So i don’t know about EGL things and what’s needed and what not…

If you switch to Mesa drivers, Frame should work.

How would i do that ? Uninstalling the proprietary nvidia drivers?

Last q (and sorry for being such a noob):

Is there a way using mesa within frame while having the nvidia drivers installed?

Basically i just want to make the snap work. If it would use nvidia drivers it would be awesome(!) if not it would be ok too (while i need nvidia drivers installed for other things running on that machine).

Thanks again for your time and effort in helping newbies like me

PS: it might was misleading in my OP: I just wanted to say I haven’t done any changes in the snapcraft.yml regarding nvidia and not that i don’t want to change it its just i don’t know what to change

alan_g · June 8, 2022, 8:08am

It is all software so, in theory, changes could be made to the snap. But, in practice, I have the impression you don’t have the knowledge to make those changes.

It requires work to identify the necessary changes, and as nobody has done the work yet there’s no “recipe” of changes. If we knew exactly what to do we would have done it.

That would work. Alternatively, is yours a hybrid system? You could just switch GPUs:

prime-select intel

jogi · June 20, 2022, 8:52am

ok so prime-select does not work for that system and as mentioned uninstalling the nvidia drivers is not an option here, too.

but… I was able to get it working → while I am not sure how exactly I made a lot of trial&error sessions and one change I did (bc I saw it mentioned somewhere) was to set nvidia_drm modeset=1 . I am 77% sure that was the last change I did and then it started to work. it was very laggy though and therefore it is not possible to use it for the intended purpose (streaming video from a cam) but at least the page came up now instead of crashing.

While not ideal at the end we added a second system without these drivers as the kiosk is running flawless without them.

anyways thanks for your thoughts and trying to help. I really hope nvidia will be supported some day within MIR.

alan_g · June 20, 2022, 9:06am

Nvidia have been working on gbm-kms support and we expect that to be the basis for Ubuntu Frame (and likely other Mir compositors) going forward. But , just now, this is not done.

As stated above, Mir does work with Nvidia’s eglstream-kms API, but that support isn’t part of the Ubuntu Frame snap. There is an experimental branch of Ubuntu Frame with egmstream support and that can be installed with:

snap install --channel edge/pr43 ubuntu-frame

If it works for you, that would be interesting, but given Nvidia’s work on the standard gbm-kms approach we are unlikely to integrate this into Frame proper.