Correct way to do Nvidia Vulkan?

snapcraft
1

I am maintaining 2 snaps that make use of Vulkan, adding vulkan support for AMD and Intel GPU’s is a straightforward addition to the snapcraft.yaml:

Adding the following layouts:

layout:
  /usr/share/vulkan:
    symlink: $SNAP/usr/share/vulkan
  /usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libvulkan_intel.so:
    symlink: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libvulkan_intel.so
  /usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libvulkan_radeon.so:

Adding to staged packages:

stage-packages:
    - libvulkan1
    - mesa-vulkan-drivers
    - vulkan-utils

However for Nvidia I haven’t found very clear instructions on how to add this or what the correct approach is?

What I was previously doing for my snaps to get Vulkan working on Nvidia was to stage the entire nvidia driver (nvidia-driver-400). This got Vulkan working but added around 150MB to the snap, also I’m not sure if there are downsides to this approach if a user is using a different driver version on their machine? I went with this approach after looking at the documentation here (Adding OpenGL/GPU support to a snap) mentioning to stage the GPU drivers. Using this approach I also noticed when selecting the Vulkan device in my snap it would show the GPU twice:
Screenshot%20from%202020-10-03%2016-58-10

Recently I tried an alternative approach of setting VK_ICD_FILENAMES: "/var/lib/snapd/lib/vulkan/icd.d/nvidia_icd.json:$SNAP/usr/share/vulkan/icd.d/radeon.x86_64.json:$SNAP/usr/share/vulkan/icd.d/intel_icd.x86_64.json" as an environment variable, which also got Vulkan to work. The issue with these filenames is that I dont know how to avoid having the x86_64 arch hardcoded for the Intel and Radeon ICD filenames. Using this approach I don’t see the duplicate Vulkan device when selecting the adapter:
Screenshot%20from%202020-10-03%2017-00-26
If I use this approach I’m assuming this won’t work on an Ubuntu Core device due to the absence of Nvidia drivers?

For both approaches I also added the following layout, to follow the pattern with the AMD and Intel drivers:

  /usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libGLX_nvidia.so.0:
    symlink: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libGLX_nvidia.so.0

But for the second approach I tried removing this to see if it would have any effect and it seems Vulkan works fine after the removal. I don’t know if removing this would have any unseen drawbacks.

So basically I’m just looking for clarification on what the correct approach for Nvidia Vulkan is, whether its one of the approaches mentioned above or something else. Also could the documentation for GPU support be expanded to mention setting up Vulkan?

3 Likes
2

Thank you @Nightmayr for this post, I found it extremely helpful when adding Vulkan support to the OBS Studio snap; the NvFBC plugin requires Vulkan.

While the OBS Studio snap is currently amd64 only, I thought I’d tackle the multi-architecture ICD path issue you described by way of a thank you :slightly_smiling_face:

I have a wrapper for OBS as the last script in the command-chain: It looks like this:

apps:
  obs-demo:
    command: usr/bin/obs --multi
    command-chain:
      - snap/command-chain/desktop-launch
      - bin/obs-wrapper

I added the following to obs-wrapper to export architecture-specific VK_ICD_FILENAMES, like this:

# Export the Vulkan ICD filename paths
#  - There is no Intel ICD for ARM
if [ "$SNAP_ARCH" = "amd64" ]; then
  export VK_ICD_FILENAMES="/var/lib/snapd/lib/vulkan/icd.d/nvidia_icd.json:$SNAP/usr/share/vulkan/icd.d/radeon.x86_64.json:$SNAP/usr/share/vulkan/icd.d/intel_icd.x86_64.json"
elif [ "$SNAP_ARCH" = "i386" ]; then
  export VK_ICD_FILENAMES="/var/lib/snapd/lib/vulkan/icd.d/nvidia_icd.json:$SNAP/usr/share/vulkan/icd.d/radeon.i686.json:$SNAP/usr/share/vulkan/icd.d/intel_icd.i686.json"
elif [ "$SNAP_ARCH" = "armhf" ]; then
  export VK_ICD_FILENAMES="/var/lib/snapd/lib/vulkan/icd.d/nvidia_icd.json:$SNAP/usr/share/vulkan/icd.d/radeon.armv7l.json"
elif [ "$SNAP_ARCH" = "arm64" ]; then
  export VK_ICD_FILENAMES="/var/lib/snapd/lib/vulkan/icd.d/nvidia_icd.json:$SNAP/usr/share/vulkan/icd.d/radeon.aarch64.json"
fi

Here is the commit where I added Vulkan support and the NvFBC plugin to the OBS Studio snapcraft.yaml; I hope others find it useful.

6 Likes