Qt app with no-system-libraries and without

sektorct · November 9, 2018, 11:33pm

I checked all the dependencies of the application, and snapcraft.yaml looks like this:

apps:
    managetime:
        command: bin/projecttimer
        plugs: 
            - home
            - unity7
            - x11
            - opengl
            - pulseaudio
            - wayland
            - network
            - desktop
            - desktop-legacy
        environment:
            XDG_RUNTIME_DIR: /run/user/1000
            __EGL_VENDOR_LIBRARY_DIRS: $SNAP/etc/glvnd/egl_vendor.d:$SNAP/usr/share/glvnd/egl_vendor.d
            # XKB config
            XKB_CONFIG_ROOT: $SNAP/usr/share/X11/xkb
            # Qt Platform to Wayland
            #QT_QPA_PLATFORM: wayland
            QT_QPA_PLATFORM: xcb
            QTCHOOSER_NO_GLOBAL_DIR: 1
            QT_SELECT: snappy-qt5
            # Qt Modules
            LD_LIBRARY_PATH: $SNAP/Qt/5.11.0/gcc_64/lib:$LD_LIBRARY_PATH
            QT_PLUGIN_PATH: $SNAP/Qt/5.11.0/gcc_64/plugins
            QML2_IMPORT_PATH: $QML2_IMPORT_PATH:$SNAP/Qt/5.11.0/gcc_64/qml
            QT_QPA_PLATFORM_PLUGIN_PATH: $SNAP/Qt/5.11.0/gcc_64/plugins/platforms
            # XDG Config
            XDG_CONFIG_DIRS: $SNAP/etc/xdg:$SNAP/usr/xdg:$XDG_CONFIG_DIRS
            XDG_DATA_DIRS: $SNAP/usr/share:$XDG_DATA_DIRS
            # Font Config and themes
            FONTCONFIG_PATH: $SNAP/etc/fonts/conf.d
            FONTCONFIG_FILE: $SNAP/etc/fonts/fonts.conf
            # Tell libGL where to find the drivers
            LIBGL_DRIVERS_PATH: $SNAP/usr/lib/$ARCH/dri
parts:
    managetime:
        plugin: dump
        source: scripts
        source-type: local
        build-attributes: 
            - keep-execstack
            - no-system-libraries

    integration:
        plugin: nil
        stage-packages:
            - libc6
            - zlib1g
            - libstdc++6
            - libgcc1
            - libglib2.0-0
            - libpcre3
            - libxext6
            - libx11-6
            - libxcb1
            - libxau6
            - libxdmcp6
            - libbsd0
            - libxdamage1
            - libxfixes3
            - libx11-xcb1
            - libxcb-glx0
            - libxcb-dri3-0
            - libxcb-present0
            - libxcb-dri2-0
            - libxxf86vm1
            - libxshmfence1
            - libxcb-sync1
            - libpulse-mainloop-glib0
            - libpulse0
            - libsystemd0
            - liblzma5
            - liblz4-1
            - libgcrypt20
            - libgpg-error0
            - libwrap0
            - libsndfile1
            - libflac8
            - libogg0
            - libvorbis0a
            - libvorbisenc2
            - libasyncns0
            - libdbus-1-3
            - libfontconfig1
            - libfreetype6
            - libpng16-16
            - libexpat1
            - libxi6
            - libxrender1
            - libdrm2
            - libwayland-client0
            - libwayland-cursor0
            - libffi6
            - libxkbcommon0
            - libxcomposite1
            - libgl1
            - libgl1-mesa-dri
            - libgl1-mesa-glx

If I install the snap package in the newly installed system, I see such errors:
libGL error: unable to load driver: swrast_dri.so
libGL error: failed to load driver: swrast
Unrecognized OpenGL version
Unrecognized OpenGL version
QGLXContext: Failed to create dummy context
QSoundEffect(pulseaudio): Error decoding source
QSoundEffect(pulseaudio): Error decoding source
Unrecognized OpenGL version
Unrecognized OpenGL version

But if I install the package in the system in which I built this package, I get such errors:

libGL error: unable to load driver: radeonsi_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: radeonsi
libGL error: unable to load driver: radeonsi_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: radeonsi
libGL error: unable to load driver: swrast_dri.so
libGL error: failed to load driver: swrast
QSoundEffect(pulseaudio): Error decoding source
QSoundEffect(pulseaudio): Error decoding source
Buffer creation failed
Buffer creation failed
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram::uniformLocation(matrix): shader program is not linked
QOpenGLShaderProgram::uniformLocation(opacity): shader program is not linked
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram::uniformLocation(qt_Matrix): shader program is not linked
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram::uniformLocation(matrix): shader program is not linked
QOpenGLShaderProgram::uniformLocation(textureScale): shader program is not linked
QOpenGLShaderProgram::uniformLocation(color): shader program is not linked
QOpenGLShaderProgram::uniformLocation(alphaMin): shader program is not linked
QOpenGLShaderProgram::uniformLocation(alphaMax): shader program is not linked
QOpenGLShaderProgram::uniformLocation(fontScale): shader program is not linked
QOpenGLShaderProgram::uniformLocation(vecDelta): shader program is not linked
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed:

For me, the cause of these errors is completely unclear((

Lin-Buo-Ren · November 11, 2018, 3:12am

Verify if the libraries depended by radeonsi_dri.so are staged.

sektorct · November 11, 2018, 7:26pm

Yes, all these drivers are present in the package, they are located in the right places.
That is why the cause of this error is not clear.

ls -l /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/
total 138505
-rw-r--r-- 1 root root  9685064 lis 10 00:16 i915_dri.so
-rw-r--r-- 1 root root  9685064 lis 10 00:16 i965_dri.so
-rw-r--r-- 1 root root 11674960 lis 10 00:16 kms_swrast_dri.so
-rw-r--r-- 1 root root 11674960 lis 10 00:16 nouveau_dri.so
-rw-r--r-- 1 root root  9685064 lis 10 00:16 nouveau_vieux_dri.so
-rw-r--r-- 1 root root  9685064 lis 10 00:16 r200_dri.so
-rw-r--r-- 1 root root 11674960 lis 10 00:16 r300_dri.so
-rw-r--r-- 1 root root 11674960 lis 10 00:16 r600_dri.so
-rw-r--r-- 1 root root  9685064 lis 10 00:16 radeon_dri.so
-rw-r--r-- 1 root root 11674960 lis 10 00:16 radeonsi_dri.so
-rw-r--r-- 1 root root 11674960 lis 10 00:16 swrast_dri.so
-rw-r--r-- 1 root root 11674960 lis 10 00:16 virtio_gpu_dri.so
-rw-r--r-- 1 root root 11674960 lis 10 00:16 vmwgfx_dri.so

ogra · November 11, 2018, 10:05pm

this must be:

LIBGL_DRIVERS_PATH: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/dri

(though with binary nvidia drivers you want it to be /var/lib/snapd/lib/gl … which i think would require a wrapper to switch between free and closed driver)

sektorct · November 11, 2018, 10:30pm

ogra:

this must be:
LIBGL_DRIVERS_PATH: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/dri
(though with binary nvidia drivers you want it to be /var/lib/snapd/lib/gl … which i think would require a wrapper to switch between free and closed driver)

I changed the parameter as you advised.
On VirtualBox I was able to run the application.
I have a graphic card(Radeon Vega 64, installed driver amdgpu-pro) on my computer, and it seems there is a new problem.

/usr/share/libdrm/amdgpu.ids: No such file or directory
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: radeonsi
/usr/share/libdrm/amdgpu.ids: No such file or directory
mesa: for the -simplifycfg-sink-common option: may only occur zero or one times!
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: radeonsi
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
QSoundEffect(pulseaudio): Error decoding source
QSoundEffect(pulseaudio): Error decoding source
Buffer creation failed
Buffer creation failed
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed: 
""
QOpenGLShaderProgram: could not create shader program
QOpenGLShader: could not create shader
QOpenGLShader: could not create shader
shader compilation failed:

How can this situation be solved?

Lin-Buo-Ren · November 12, 2018, 2:59am

I mean the libraries depended by radeonsi_dri.so, it is very common that a shared library depends on other shared libraries, to verify this you have to run the following commands:

$ snap run --shell managetime
$ eval "$(head --lines=-1 $SNAP/command-managetime.wrapper)"
$ ldd "$SNAP"/usr/lib/x86_64-linux-gnu/dri/radeonsi_dri.so

and verify if there’s any ‘not found’ library entries.

sektorct · November 12, 2018, 7:48am

It seems like all the libraries are found.

ldd "$SNAP"/usr/lib/x86_64-linux-gnu/dri/radeonsi_dri.so 
        linux-vdso.so.1 =>  (0x00007ffd4538c000)
        libz.so.1 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libz.so.1 (0x00007fdc73198000)
        libglapi.so.0 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libglapi.so.0 (0x00007fdc72f66000)
        libsensors.so.4 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libsensors.so.4 (0x00007fdc72d56000)
        libpthread.so.0 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdc72b37000)
        libdl.so.2 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libdl.so.2 (0x00007fdc72932000)
        libexpat.so.1 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fdc72708000)
        libdrm_nouveau.so.2 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libdrm_nouveau.so.2 (0x00007fdc724ff000)
        libdrm_radeon.so.1 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libdrm_radeon.so.1 (0x00007fdc722f2000)
        libdrm_amdgpu.so.1 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libdrm_amdgpu.so.1 (0x00007fdc720e7000)
        libdrm.so.2 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libdrm.so.2 (0x00007fdc71ed3000)
        libelf.so.1 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libelf.so.1 (0x00007fdc71cba000)
        libLLVM-6.0.so.1 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libLLVM-6.0.so.1 (0x00007fdc6dfda000)
        libstdc++.so.6 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libstdc++.so.6 (0x00007fdc6dc17000)
        libm.so.6 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libm.so.6 (0x00007fdc6d90c000)
        libc.so.6 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libc.so.6 (0x00007fdc6d541000)
        libgcc_s.so.1 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fdc6d32a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fdc742cf000)
        libffi.so.6 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libffi.so.6 (0x00007fdc6d121000)
        libedit.so.2 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../libedit.so.2 (0x00007fdc6cee6000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fdc6ccbd000)
        libbsd.so.0 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fdc6caa7000)

Lin-Buo-Ren · November 12, 2018, 9:07am

Except for this one:

which isn’t provided by the snap, note that in the runtime the snapped application can’t access libraries that aren’t bundled in the snap (with a few exceptions which are provided by the core snap, including the C runtime library).

Although the libtinfo.so.5 library does exist in the core snap (/snap/core/current/lib/x86_64-linux-gnu/libtinfo.so.5), it is possible that it isn’t in the snapd’s whitelist of library access and thus isn’t available to the snap, please try to add libtinfo5 to the stage-packages and verify if it make the library loading work.

Lin-Buo-Ren · November 12, 2018, 9:11am

This is also a potential cause of the failure of loading the driver support library, please add libdrm-common to the stage-packages.

Note that it is likely that the library still can’t access the amdgpu.ids file since the staged file is rather located at $SNAP/usr/share/libdrm/amdgpu.ids instead of /usr/share/libdrm/amdgpu.ids, in this case you have to fix the problem via rebuilding DRM libraries with the proper DATAROOTDIR or PREFIX so that it can locate the required software resources, there are other experimental option to solve the problem as well, like sergiusens/snapcraft-preload: Use smart preloading in your snaps to avoid hard coded parts that wold fail security. and the Snap layouts - doc - snapcraft.io, though I’m not sure those are ideal or not.

/cc @sergiusens , @zyga-snapd

sektorct · November 12, 2018, 8:10pm

Yes, your advice helped. Now the library is in the snap package
libtinfo.so.5 => /snap/managetime/x1/usr/lib/x86_64-linux-gnu/dri/../../../../lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f0882965000)

sektorct · November 13, 2018, 3:20pm

Could you explain what to do with this? I did not quite understand where and what should be setting. But $SNAP/usr/share/libdrm/amdgpu.ids is in the package.

Lin-Buo-Ren · November 13, 2018, 3:30pm

The problem is that the library is hardcoded to look at /usr/share/libdrm/amdgpu.ids instead of $SNAP/usr/share/libdrm/amdgpu.ids during the configure phase (as /usr is the proper install prefix and /usr/share is the proper data lookup location for regular software in Ubuntu), you have to rebuild it with the proper resource paths to solve the problem.

In short, you have to rebuild libdrm as a new part with the --datarootdir set as $SNAP/share

sektorct · November 13, 2018, 8:42pm

It’s should be like this?

parts:
    managetime:
        plugin: dump
        source: scripts
        source-type: local
        build-attributes: 
            - keep-execstack
            - no-system-libraries
        plugin: autotools
        configflags:
            - --datarootdir=$SNAP/share
        organize:
            $SNAP/share/: /

Lin-Buo-Ren · November 14, 2018, 1:48am

No, it is about the software part called libdrm, instead of your target application.

You have to fetch libdrm’s source package, extract it to parts/part_name/src using the override-pull scriptlet, and build it with the desirable plugin and configflags.

sektorct · November 14, 2018, 12:00pm

I understand you correctly, I need to unpack the package of libdrm-common to path $SNAP/share, and after that I collect the snap package?
Because now I just added this package to stage-packages.

Lin-Buo-Ren · November 14, 2018, 3:13pm

Nope, you have to completely replace it with the version provided by the new part, as the staged Ubuntu libdrm package can’t fulfill the requirement.

The proper way should be using the applied/ubuntu/bionic branch(should be applied/ubuntu/bionic-updates instead, but it seems that the updates release pocket of the bionic suite isn’t available yet) of the libdrm package as the new part’s source and build it with the custom configflags.

sektorct · November 14, 2018, 8:43pm

I can not understand why when I install this package on a virtualbox, then there are no such problems?
But amdgpu.ids is in the snap package and in the system (/usr/share/libdrm). This is strange.

Lin-Buo-Ren · November 15, 2018, 1:39am

Because the virtual machine doesn’t have AMD graphics hardware…

The library can’t access the copy in the system and doesn’t locate the copy in the snap package.

sektorct · January 10, 2019, 12:42pm

I think that I know what the problem is, if I add after: [qt5], then this problem should be corrected.
Tell me please, for my configuration, where should I add after: [qt5] to get a package? This should be in parts?

apps:
    managetime:
        desktop: share/applications/managetime.desktop
        command: bin/projecttimer
        plugs: 
            - home
            - unity7
            - x11
            - opengl
            - pulseaudio
            - wayland
            - network
            - desktop
            - desktop-legacy
        environment:
            XDG_RUNTIME_DIR: /run/user/1000
            __EGL_VENDOR_LIBRARY_DIRS: $SNAP/etc/glvnd/egl_vendor.d:$SNAP/usr/share/glvnd/egl_vendor.d
            # XKB config
            XKB_CONFIG_ROOT: $SNAP/usr/share/X11/xkb
            # Qt Platform to Wayland
            #QT_QPA_PLATFORM: wayland
            QT_QPA_PLATFORM: xcb
            QTCHOOSER_NO_GLOBAL_DIR: 1
            QT_SELECT: snappy-qt5
            # Qt Modules
            LD_LIBRARY_PATH: $SNAP/Qt/5.11.0/gcc_64/lib:$LD_LIBRARY_PATH
            QT_PLUGIN_PATH: $SNAP/Qt/5.11.0/gcc_64/plugins
            QML2_IMPORT_PATH: $QML2_IMPORT_PATH:$SNAP/Qt/5.11.0/gcc_64/qml
            QT_QPA_PLATFORM_PLUGIN_PATH: $SNAP/Qt/5.11.0/gcc_64/plugins/platforms
            # XDG Config
            XDG_CONFIG_DIRS: $SNAP/etc/xdg:$SNAP/usr/xdg:$XDG_CONFIG_DIRS
            XDG_DATA_DIRS: $SNAP/usr/share:$XDG_DATA_DIRS
            # Font Config and themes
            FONTCONFIG_PATH: $SNAP/etc/fonts/conf.d
            FONTCONFIG_FILE: $SNAP/etc/fonts/fonts.conf
            # Tell libGL where to find the drivers
            LIBGL_DRIVERS_PATH: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/dri
parts:
    managetime:
        plugin: dump
        source: scripts
        source-type: local
        build-attributes: 
            - keep-execstack

    integration:
        plugin: nil
        stage-packages:
            - libc6
            - zlib1g
            - libstdc++6
            - libgcc1
            - libglib2.0-0
            - libpcre3
            - libxext6
            - libx11-6
            - libxcb1
            - libxau6
            - libxdmcp6
            - libbsd0
            - libxdamage1
            - libxfixes3
            - libx11-xcb1
            - libxcb-glx0
            - libxcb-dri3-0
            - libxcb-present0
            - libxcb-dri2-0
            - libxxf86vm1
            - libxshmfence1
            - libxcb-sync1
            - libpulse-mainloop-glib0
            - libpulse0
            - libsystemd0
            - liblzma5
            - liblz4-1
            - libgcrypt20
            - libgpg-error0
            - libwrap0
            - libsndfile1
            - libflac8
            - libogg0
            - libvorbis0a
            - libvorbisenc2
            - libasyncns0
            - libdbus-1-3
            - libfontconfig1
            - libfreetype6
            - libpng16-16
            - libexpat1
            - libxi6
            - libxrender1
            - libdrm2
            - libwayland-client0
            - libwayland-cursor0
            - libffi6
            - libxkbcommon0
            - libxcomposite1
            - libgl1
            - libgl1-mesa-dri
            - libgl1-mesa-glx
            - libdrm-common
            - libtinfo5

sektorct · January 11, 2019, 7:26pm

What is it mean?

Failed to get part information: Cannot find the definition for part 'qt5', required by part 'managetime'.
Remote parts are not supported with bases, so make sure that this part is defined in the `snapcraft.yaml`.
Run the same command again with --debug to shell into the environment if you wish to introspect this failure.
An error occurred when trying to execute 'sudo -i snapcraft snap' with 'multipass': returned exit code 2.