Ubuntu Core 18 /etc Read-only filesystem

giuliano0697 · April 6, 2021, 3:20pm

Hi all,
I have an issue with /etc folder in Ubuntu Core 18. I am really a beginner, so I try to explain everything I have done until now.

I took the Generic x86/KVM core-18-amd64 image from the UC website and created a bootable USB using Win32DiskImager. Then, I booted my PC from that and performed the configuration. After that, I connected remotely to it and used $ sudo passwd ubuntu-one-account to create a local user.

On another PC (having Ubuntu 18.04), I have built a snap that I want to use on UC. This snap uses an OMRON Sentech camera, that needs to perform the following commands to be correctly used:

$ sudo cp /home/username/Documents/cams/.stprofile /etc/profile.d/stprofile.sh
$ sudo cp /home/username/Documents/cams/stldlib.conf /etc/ld.so.conf.d/
$ sudo ldconfig

I tried to perform them on UC, but /etc/profile.d is not present and I cannot either perform the second command, since I get that /etc is a Read-only filesystem.

I have tried the snap on both the PC on which I have built it and on another Ubuntu 18.04 PC, and it works. Instead, after installing it in UC, I get an error about the fact that the libStApi_TL.so.1.1 library cannot be found. I am quite sure that it is due to the incomplete installation of the camera.

Therefore, is there a way to avoid the error about the /etc folder permissions?
I have tried to use chmod, even as root ($ sudo su), but I get the same error.
Please, try to explain me what to do in the simplest possible way.

Thank you in advance!

ijohnson · April 6, 2021, 4:12pm

Neither of these paths are part of writable-paths, so in order to persist files on /etc/ outside of the snap’s mount namespace you would need to get those directories add to writable-paths.

Can you elaborate on what you need these directories for with your snap? What sort of content is in the files you are trying to install there? If you just need to adjust the load path of your snap to load libStAPI_TL.so.1.1, you can use the environment variable LD_LIBRARY_PATH to add directories to the search path which doesn’t have any of the issues with read only filesystems. I’m not sure what your config in profile.d is doing, so I’m not sure if there are other workarounds there.

Additionally, if these directories are only needed for your snap application in particular and don’t need to be shared across multiple snaps on the system, you can use layouts to expose just these directories for your snap application alone, something like this in the snapcraft.yaml:

layout:
   /etc/profile.d:
    bind: $SNAP_DATA/etc/profile.d
  /etc/ld.so.conf.d:
    bind: $SNAP_DATA/etc/ld.so.conf.d

and then have your snap in an install hook copy the default config you want in those files from $SNAP (which is the read-only part of your snap that you get when you build the snap) to $SNAP_DATA/etc/… which is writable, but initially empty when your snap gets installed.

This sounds like an issue from not having the library search path setup, take a look at the above suggestion about trying LD_LIBRARY_PATH.

giuliano0697 · April 7, 2021, 8:30am

Hi,
actually I had alredy tried to put the /home/username/Documents/cams/lib path (where the “missing” library libStApi_TL.so.1.1 is located) inside $LD_LIBRARY_PATH by adding the following line in the /home/username/.bashrc file (which is analogous to the content of the first file that I should copy in /etc):
export LD_LIBRARY_PATH="/home/username/Documents/cams/lib:$LD_LIBRARY_PATH"

Moreover, still in the .bashrc, I had also tried to put the content of the second file that I should copy in the /etc folder, in particular:

STAPI_ROOT_PATH=/home/username/Documents/cams
GENICAM_GENTL64_PATH=/home/username/Documents/cams/lib:$GENICAM_GENTL64_PATH
LIBRARY_PATH=$STAPI_ROOT_PATH/lib:$STAPI_ROOT_PATH/lib/GenICam:$LIBRARY_PATH
LD_LIBRARY_PATH=$STAPI_ROOT_PATH/lib:$STAPI_ROOT_PATH/lib/GenICam:$LD_LIBRARY_PATH
export GENICAM_GENTL64_PATH STAPI_ROOT_PATH LIBRARY_PATH LD_LIBRARY_PATH

Although, I still got that error:
ImportError: libStApi_TL.so.1.1: cannot open shared object file: No such file or directory

Let me also add that the camera installation (.run file) asks me to also install the following packages using apt (which, if I understood well, I cannot use since I am on Ubuntu Core): qt5-default, qtchooser, libpng16-16, libturbojpeg
So, I tried to include them among the stage-packages of the python plugin in the snapcraft.yaml, and indeed I see them in the following folder of the snap (also in UC):
/snap/snap-name/8/usr/share/doc
Therefore, I also tried to include this path in $LD_LIBRARY_PATH, still from the .bashrc file, but again the installer tells me of not being able to find them (so I suppose that it just checks for their presence in the /usr/share/doc folder… Which of course is Read-only too, so I cannot copy the libraries there from the snap folder).

Another thing that I noticed is that, during the building of the snap through the snapcraft command, I get a warning about the fact that the missing library cannot be found:

This part is missing libraries that cannot be satisfied with any available stage-packages known to snapcraft:
- libStApi_IP.so.1.1
- libStApi_TL.so.1.1

I have not given much importance to it up to now, since the snap is able to find those libraries on Ubuntu 18.04, so I suspect more that the issue is due to the missing packages during the installation.

I hope to have been clearer now. If you need some more information to help me solving the issue I am here! Thank you in advance @ijohnson

ogra · April 7, 2021, 8:49am

just make sure snapcraft can access these libs during build of your snap and use a part to copy them into $SNAPCRAFT_PART_INSTALL/lib (or usr/lib) … this dir is automatically in the LD_LIBRARY_PATH of your snap at execution time …

giuliano0697 · April 7, 2021, 9:46am

Could you please explain me how to do this a bit more in detail?
My snapcraft.yaml is the following up to now:

name: snap-name
version: xxxx
summary: xxxx
description: xxxx

base: core18
grade: devel
confinement: devmode

apps:
  snap-name:
    command: bin/snap-name
    plugs: [camera,home,display-control,x11]

parts:
  snap-name:
    plugin: python
    python-version: python3
    source: .
    requirements: [./requirements.txt]
    stage-packages: [x11-utils,python3-tk,python3-distutils,libturbojpeg,qt5-default,qtchooser,libpng16-16,libdouble-conversion1,libdrm-dev,libegl1-mesa-dev,libgl1-mesa-dev,
                     libgles1,libgles2-mesa-dev,libglu1-mesa-dev,libglvnd-core-dev,libglvnd-dev, libopengl0,libpthread-stubs0-dev,libqt5concurrent5,libqt5core5a,libqt5dbus5,
                     libqt5gui5,libqt5network5,libqt5opengl5,libqt5opengl5-dev,libqt5printsupport5,libqt5sql5,libqt5sql5-sqlite,libqt5svg5,libqt5test5,libqt5widgets5,
                     libqt5xml5,libwayland-bin,libwayland-dev,libx11-dev,libx11-doc,libx11-xcb-dev,libxau-dev,libxcb-dri2-0-dev,libxcb-dri3-dev,libxcb-glx0-dev,
                     libxcb-present-dev,libxcb-randr0-dev,libxcb-render0-dev,libxcb-shape0-dev,libxcb-sync-dev,libxcb-xfixes0-dev,libxcb-xinerama0,libxcb1-dev,libxdamage-dev,
                     libxdmcp-dev,libxext-dev,libxfixes-dev,libxshmfence-dev,libxxf86vm-dev,mesa-common-dev,qt5-gtk-platformtheme,qt5-qmake,qt5-qmake-bin,qtbase5-dev,
                     qtbase5-dev-tools,qttranslations5-l10n,x11proto-core-dev,x11proto-damage-dev,x11proto-dev,x11proto-fixes-dev,x11proto-xext-dev,x11proto-xf86vidmode-dev,
                     xorg-sgml-doctools,xtrans-dev]

In the stage-packages of the python plugin I should have put all the apt packages that the camera installation procedure installs.

The content of the folder where the missing library is placed is the following ($ ls -l):

drwxr-xr-x 2 root root    4096 Mar 24 14:58 GenICam
lrwxrwxrwx 1 root root      17 Mar 24 14:58 libStApi_C.so -> libStApi_C.so.1.1
lrwxrwxrwx 1 root root      19 Mar 24 14:58 libStApi_C.so.1.1 -> libStApi_C.so.1.1.2
-rwxr-xr-x 1 root root 1111160 Mar 24 14:58 libStApi_C.so.1.1.2
lrwxrwxrwx 1 root root      22 Mar 24 14:58 libStApi_GUI_qt.so -> libStApi_GUI_qt.so.1.1
lrwxrwxrwx 1 root root      24 Mar 24 14:58 libStApi_GUI_qt.so.1.1 -> libStApi_GUI_qt.so.1.1.2
-rwxr-xr-x 1 root root 2170136 Mar 24 14:58 libStApi_GUI_qt.so.1.1.2
lrwxrwxrwx 1 root root      18 Mar 24 14:58 libStApi_IP.so -> libStApi_IP.so.1.1
lrwxrwxrwx 1 root root      20 Mar 24 14:58 libStApi_IP.so.1.1 -> libStApi_IP.so.1.1.2
-rwxr-xr-x 1 root root 2279864 Mar 24 14:58 libStApi_IP.so.1.1.2
lrwxrwxrwx 1 root root      18 Mar 24 14:58 libStApi_TL.so -> libStApi_TL.so.1.1
lrwxrwxrwx 1 root root      20 Mar 24 14:58 libStApi_TL.so.1.1 -> libStApi_TL.so.1.1.2
-rwxr-xr-x 1 root root  721016 Mar 24 14:58 libStApi_TL.so.1.1.2
lrwxrwxrwx 1 root root      17 Mar 24 14:58 libstgentl.cti -> libstgentl.so.1.5
lrwxrwxrwx 1 root root      17 Mar 24 14:58 libstgentl.so -> libstgentl.so.1.5
lrwxrwxrwx 1 root root      19 Mar 24 14:58 libstgentl.so.1.5 -> libstgentl.so.1.5.0
-rwxr-xr-x 1 root root  737016 Mar 24 14:58 libstgentl.so.1.5.0
lrwxrwxrwx 1 root root      15 Mar 24 14:58 libusb-1.0.so -> libusb-1.0.so.0
lrwxrwxrwx 1 root root      19 Mar 24 14:58 libusb-1.0.so.0 -> libusb-1.0.so.0.2.0
-rwxr-xr-x 1 root root  148136 Mar 24 14:58 libusb-1.0.so.0.2.0

And its GenICam subfolder has these things:

-rwxr-xr-x 1 root root  872056 Mar 24 14:58 libFirmwareUpdate_gcc48_v3_2.so
lrwxrwxrwx 1 root root      31 Mar 24 14:58 libFirmwareUpdate.so -> libFirmwareUpdate_gcc48_v3_2.so
-rwxr-xr-x 1 root root  101696 Mar 24 14:58 libGCBase_gcc48_v3_2.so
lrwxrwxrwx 1 root root      23 Mar 24 14:58 libGCBase.so -> libGCBase_gcc48_v3_2.so
-rwxr-xr-x 1 root root 4921136 Mar 24 14:58 libGenApi_gcc48_v3_2.so
lrwxrwxrwx 1 root root      23 Mar 24 14:58 libGenApi.so -> libGenApi_gcc48_v3_2.so
-rwxr-xr-x 1 root root  291304 Mar 24 14:58 liblog4cpp_gcc48_v3_2.so
lrwxrwxrwx 1 root root      24 Mar 24 14:58 liblog4cpp.so -> liblog4cpp_gcc48_v3_2.so
-rwxr-xr-x 1 root root   23040 Mar 24 14:58 libLog_gcc48_v3_2.so
lrwxrwxrwx 1 root root      20 Mar 24 14:58 libLog.so -> libLog_gcc48_v3_2.so
-rwxr-xr-x 1 root root   51800 Mar 24 14:58 libMathParser_gcc48_v3_2.so
lrwxrwxrwx 1 root root      27 Mar 24 14:58 libMathParser.so -> libMathParser_gcc48_v3_2.so
-rwxr-xr-x 1 root root  109576 Mar 24 14:58 libNodeMapData_gcc48_v3_2.so
lrwxrwxrwx 1 root root      28 Mar 24 14:58 libNodeMapData.so -> libNodeMapData_gcc48_v3_2.so
-rwxr-xr-x 1 root root 1428640 Mar 24 14:58 libXmlParser_gcc48_v3_2.so
lrwxrwxrwx 1 root root      26 Mar 24 14:58 libXmlParser.so -> libXmlParser_gcc48_v3_2.so

The missing library is used by the following import that is used inside the Python code that the snap executes (I put its .whl file inside the requirements.txt, since can be installed with pip):
import stapipy

ogra · April 7, 2021, 10:36am

can you edit your post and put triple backticks ``` above and below the pasted snapcraft.yaml content ? else the indendation and formatting are lost and it becomes completely unreadable …

giuliano0697 · April 7, 2021, 10:47am

Done, sorry but I did not really know how to do it

ogra · April 7, 2021, 10:58am

create a dir called “extra-libs” in your source tree, copy all libst* and libSt* files into it … then add a part to the end of your snapcraft.yaml like i did below …

name: snap-name
version: xxxx
summary: xxxx
description: xxxx

base: core18
grade: devel
confinement: devmode

apps:
  snap-name:
    command: bin/snap-name
    plugs:
      - camera
      - home
      - display-control
      - x11

parts:
  snap-name:
    plugin: python
    python-version: python3
    source: .
    requirements: [./requirements.txt]
    stage-packages:
      - x11-utils
      - python3-tk
      - python3-distutils
      - libturbojpeg
      - qt5-default
      - qtchooser
      - libpng16-16
      - libdouble-conversion1
      - libdrm-dev
      - libegl1-mesa-dev
      - libgl1-mesa-dev
      - libgles1
      - libgles2-mesa-dev
      - libglu1-mesa-dev
      - libglvnd-core-dev
      - libglvnd-dev
      - libopengl0
      - libpthread-stubs0-dev
      - libqt5concurrent5
      - libqt5core5a
      - libqt5dbus5
      - libqt5gui5
      - libqt5network5
      - libqt5opengl5
      - libqt5opengl5-dev
      - libqt5printsupport5
      - libqt5sql5
      - libqt5sql5-sqlite
      - libqt5svg5
      - libqt5test5
      - libqt5widgets5
      - libqt5xml5
      - libwayland-bin
      - libwayland-dev
      - libx11-dev
      - libx11-doc
      - libx11-xcb-dev
      - libxau-dev
      - libxcb-dri2-0-dev
      - libxcb-dri3-dev
      - libxcb-glx0-dev
      - libxcb-present-dev
      - libxcb-randr0-dev
      - libxcb-render0-dev
      - libxcb-shape0-dev
      - libxcb-sync-dev
      - libxcb-xfixes0-dev
      - libxcb-xinerama0
      - libxcb1-dev
      - libxdamage-dev
      - libxdmcp-dev
      - libxext-dev
      - libxfixes-dev
      - libxshmfence-dev
      - libxxf86vm-dev
      - mesa-common-dev
      - qt5-gtk-platformtheme
      - qt5-qmake
      - qt5-qmake-bin
      - qtbase5-dev
      - qtbase5-dev-tools
      - qttranslations5-l10n
      - x11proto-core-dev
      - x11proto-damage-dev
      - x11proto-dev
      - x11proto-fixes-dev
      - x11proto-xext-dev
      - x11proto-xf86vidmode-dev
      - xorg-sgml-doctools
      - xtrans-dev
  extra-libs:
    plugin: nil
    source: extra-libs
    override-build: |
      mkdir -p $SNAPCRAFT_PART_INSTALL/lib
      cp -av lib* $SNAPCRAFT_PART_INSTALL/lib/

oh, and remove all the -dev packages from your stage package list, they only contain headers used for building something from source and just make your snap huge … as i understand you are packaging an existing binary here …

giuliano0697 · April 8, 2021, 9:41am

Thank you very much @ogra
Now the snap works also on Ubuntu Core!

Actually, I had to slightly modify the snapcraft.yaml, since I still got some warnings about missing libraries, that I am quite sure that were due to the “relation” between the libraries that were set by the two parts that you told me to define. In particular, I solved by putting all the libraries (also the ones contained in the GenICam folder) in /extra-libs, adding an “after” relation in the snapcraft.yaml and some stage-packages in the extra-libs part:

name: snap-name
version: xxxx
summary: xxxx
description: xxxx

base: core18
grade: devel
confinement: devmode

apps:
  snap-name:
    command: bin/snap-name
    plugs:
      - camera
      - home
      - display-control
      - x11

parts:
  snap-name:
    plugin: python
    python-version: python3
    source: .
    after: [extra-libs]
    requirements: [./requirements.txt]
    stage-packages:
      - x11-utils
      - python3-tk
      - python3-distutils
      - libturbojpeg
      - qt5-default
      - qtchooser
      - libpng16-16
      - libdouble-conversion1
      - libgles1
      - libopengl0
      - libqt5concurrent5
      - libqt5core5a
      - libqt5dbus5
      - libqt5gui5
      - libqt5network5
      - libqt5opengl5
      - libqt5printsupport5
      - libqt5sql5
      - libqt5sql5-sqlite
      - libqt5svg5
      - libqt5test5
      - libqt5widgets5
      - libqt5xml5
      - libwayland-bin
      - libx11-doc
      - libxcb-xinerama0
      - qt5-gtk-platformtheme
      - qt5-qmake
      - qt5-qmake-bin
      - qtbase5-dev-tools
      - qttranslations5-l10n
      - xorg-sgml-doctools
  extra-libs:
    plugin: nil
    source: extra-libs
    override-build: |
      mkdir -p $SNAPCRAFT_PART_INSTALL/lib
      cp -av lib* $SNAPCRAFT_PART_INSTALL/lib/
    stage-packages:
      - libturbojpeg
      - qt5-default
      - qtchooser
      - libpng16-16