Is slimmer snapd-glib possible

busybox · May 12, 2020, 2:29pm

@robert.ancell @jamesh snapd-glib is 15M in size. Is there a possibility to have a slimmer lib?

mborzecki · May 12, 2020, 6:45pm

The actual lib is <400kB. Where did you find 15MB?

jamesh · May 13, 2020, 2:48am

Presumably he means "everything pulled in when adding libsnapd-glib1 to stage-packages" (hopefully you’re not staging the development packages too?). That’s likely to pull in a number of files you don’t need in your snap: possibly some utilities, man pages, and other documentation. You can probably filter some of that stuff out via the stage: section of your part.

You should be able to use GLib from the base snap, but would likely be pulling in libsoup and libjson-glib plus any of their dependencies not in the base. If you can’t share your Snapcraft project file, It would be helpful to see a list of the new files in your snap after you added the snapd-glib dependency.

busybox · May 13, 2020, 2:55am

That’s correct. I should have been more explicit in saying that it is the snap size which is 15M with following stage-package and 8K without it.

No

Here is the snapcraft.yaml file:

name: hw-wtdog
version: '0.0.2.4'
summary: HW watchdog timer snap
description: HW watchdog snap to pet hw watchdog every 2 minutes
confinement: strict
grade: devel
base: core18

apps:
    runner:
        command: setpet
        daemon: simple
        restart-condition: always
        passthrough:
            # Run every 2 minutes
            timer: 0:00-24:00/720
        plugs: [snapd-control]

parts:
    setpet:
        plugin: cmake
        source: src/
        configflags:
          - -DBUILD_TESTS:BOOL=OFF
          - -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
          - -DLWS_SNAPD_GLIB_INCLUDE_DIRS=/usr/include/snapd-glib
          - -DLWS_GLIB2_INCLUDE_DIRS1=/usr/include/glib-2.0
          - -DLWS_GLIB2_INCLUDE_DIRS2=/usr/lib/x86_64-linux-gnu/glib-2.0/include
        build-packages:
         - libsnapd-glib-dev
        stage-packages:
          - libsnapd-glib1

jamesh · May 13, 2020, 3:19am

I would start off by adding a stage: section to your part to narrow down what files to include. Use something like this as a starting point:

stage:
  - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/lib*.so.*

… plus any extra lines to stage your own program.

On a side note, if you want to link with snapd-glib from a CMake project, I would suggest using CMake’s standard pkg-config support macros. Something like:

find_package(PkgConfig)
pkg_check_modules(SNAPD_GLIB REQUIRED snapd-glib)

This will define a PkgConfig::SNAPD_GLIB target that you can link your executable against.

busybox · May 13, 2020, 3:58am

Thanks. CMakefile.txt and snapcraft.yaml look much better now without hardcoded paths and lib’s.

I did:

build-packages:
 - libsnapd-glib-dev
stage-packages:
  - libsnapd-glib1
stage:
    - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/lib*.so.*
    - usr/bin/setpet

and

target_link_libraries(setpet ${SNAPD_GLIB_LIBRARIES})

But did not help. Snap size is still 15M. This is because all libs in usr/lib/$SNAPCRAFT_ARCH_TRIPLET/lib*.so.* add up to ~15M. I see in CMake o/p that my bin setpet links only against

... -o setpet -lsnapd-glib -lsoup-2.4 -ljson-glib-1.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0

Can I then remove other libs in usr/lib/$SNAPCRAFT_ARCH_TRIPLET/ in override-stage?

jamesh · May 13, 2020, 4:07am

The other libraries are likely dependencies of the ones that appear on the link line. If you run ldd on your executable, it will likely show them.

Could you provide a list of the libraries that do get included in your snap after these changes? That would help in determining what can be trimmed.

busybox · May 13, 2020, 4:31am

Here is the ldd o/p:

    linux-vdso.so.1 (0x00007fffe2520000)
    libsnapd-glib.so.1 => /usr/lib/x86_64-linux-gnu/libsnapd-glib.so.1 (0x00007f6219120000)
    libglib-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f6218e09000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6218a18000)
    libsoup-2.4.so.1 => /usr/lib/x86_64-linux-gnu/libsoup-2.4.so.1 (0x00007f6218725000)
    libjson-glib-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libjson-glib-1.0.so.0 (0x00007f62184fe000)
    libgio-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007f621815f000)
    libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f6217f0b000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f6217c99000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6217a7a000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f621956f000)
    libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f62176b9000)
    libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f62173b0000)
    libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f6217165000)
    libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007f6216f61000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6216d44000)
    libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f6216b1c000)
    libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f6216901000)
    libmount.so.1 => /lib/x86_64-linux-gnu/libmount.so.1 (0x00007f62166ad000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f62164a5000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f62162a1000)
    libicuuc.so.60 => /usr/lib/x86_64-linux-gnu/libicuuc.so.60 (0x00007f6215ee9000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f6215cc3000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6215925000)
    libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f621564f000)
    libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f621541d000)
    libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f6215219000)
    libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f621500e000)
    libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f6214dc1000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6214bb9000)
    libicudata.so.60 => /usr/lib/x86_64-linux-gnu/libicudata.so.60 (0x00007f6213010000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6212c87000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6212a6f000)
    libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f621286b000)
    libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f6212664000)

Hence included only the lib’s present in ldd o/p above:

     stage:
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libffi.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libgio-2.0.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libglib-2.0.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libgmodule-2.0.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libgobject-2.0.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libgssapi_krb5.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libicudata.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libicuuc.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libjson-glib-1.0.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libkrb5.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libkrb5support.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libsnapd-glib.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libsoup-2.4.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libsqlite3.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libxml2.so*
        - usr/bin/setpet

But snap size got trimmed by 3M only.

-rw-r--r-- 1 root root 12M May 12 21:27 hw-wtdog_0.0.2.4_amd64.snap

busybox · May 13, 2020, 5:21am

@jamesh do you see anything more possible on this matter. Appreciate your feedback on this.

jamesh · May 13, 2020, 6:12am

As a first step, you don’t need to include any libraries provided by the base snap. So libffi, libgio, libglib, libgmodule, libgobject, libgssapi_krb5, libkrb5, libkrb5support, and libsqlite can probably all be safely omitted.

ICU is probably the main offender though: it contains large data tables derived from the Unicode spec. I do see it in the ldd output for libsnapd-glib.so.1 though, so it isn’t as simple as omitting it.

Looking at the libraries library dependencies, it looks like this gets pulled in via:

libsnapd-glib.so.1
- libsoup-2.4.so.1
  - libxml2.so.2
    - libicuuc.so.60
      - libicudata.so.60

There isn’t an easy solution for this besides building your own copy of libxml2 without ICU support. You could probably still use the other libraries from the distro on top of the custom libxml2 in this case: I don’t believe libsoup uses any of the ICU dependent APIs. Furthermore, I doubt we’re even hitting any of the libsoup code paths that call into libxml in the first place.

busybox · May 13, 2020, 9:17am

Thanks @jamesh with reduced set of libraries now the size stands at 9.6M.
Have to include following at the least:

    stage-packages:
      - libicu60
      - libsoup2.4-1
      - libxml2
      - libsnapd-glib1
    stage:
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libicuuc.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libicudata.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libjson-glib-1.0.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libsnapd-glib.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libsoup-2.4.so*
        - usr/lib/$SNAPCRAFT_ARCH_TRIPLET/libxml2.so*
        - usr/bin/setpet

without which I see:

2020-05-13T07:14:00Z hw-wtdog.runner[2975]: /snap/hw-wtdog/x1/usr/bin/setpet: error while loading shared libraries: libsoup-2.4.so.1: cannot open shared object file: No such file or directory
2020-05-13T07:17:00Z hw-wtdog.runner[3543]: /snap/hw-wtdog/x2/usr/bin/setpet: error while loading shared libraries: libicudata.so.60: cannot open shared object file: No such file or directory
2020-05-13T07:36:03Z hw-wtdog.runner[5465]: /snap/hw-wtdog/x5/usr/bin/setpet: error while loading shared libraries: libicuuc.so.60: cannot open shared object file: No such file or directory