How to include a go lib which seems missing

snapcraft
1

I’m building a go package.

I’m using the go plugin, which seems to build the source fine and ships binaries in the built snap.

Here is the Makefile of the sources I’m building: erigon/Makefile at main · ledgerwatch/erigon · GitHub

However, after the build, it seems to be missing a lib from the build.

2024-07-07 00:25:38.957 libc.so.6 with original path /snap/core22/current/lib/x86_64-linux-gnu/libc.so.6 found on '/snap/core22/current/lib/x86_64-linux-gnu/libc.so.6' in base True                                                                                                     
2024-07-07 00:25:38.957 dynamic linker name is: 'ld-linux-x86-64.so.2'                                                                                                                                                                                                                   
2024-07-07 00:25:38.957 Lint warnings:                                                                                                                                                                                                                                                   
2024-07-07 00:25:38.957 - library: bin/capcli: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                     
2024-07-07 00:25:38.957 - library: bin/caplin: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                     
2024-07-07 00:25:38.957 - library: bin/devnet: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                     
2024-07-07 00:25:38.957 - library: bin/erigon: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                     
2024-07-07 00:25:38.957 - library: bin/erigoncustom: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                               
2024-07-07 00:25:38.958 - library: bin/evm: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                        
2024-07-07 00:25:38.958 - library: bin/hack: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                       
2024-07-07 00:25:38.958 - library: bin/integration: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                
2024-07-07 00:25:38.958 - library: bin/pics: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                       
2024-07-07 00:25:38.958 - library: bin/rpcdaemon: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                  
2024-07-07 00:25:38.958 - library: bin/sentinel: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                   
2024-07-07 00:25:38.958 - library: bin/sentry: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                     
2024-07-07 00:25:38.958 - library: bin/silkworm_api: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                               
2024-07-07 00:25:38.958 - library: bin/snapshots: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                  
2024-07-07 00:25:38.958 - library: bin/state: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                      
2024-07-07 00:25:38.958 - library: bin/txpool: missing dependency 'libsilkworm_capi.so'. (https://snapcraft.io/docs/linters-library)                                                                                                                                                     
2024-07-07 00:25:38.958 pack_snap: output=None, compression='xz'

There seems to be some attempt to do something with this lib, but somehow it isn’t making it into the snap:

2024-07-07 01:51:48.129 :: 2024-07-07 01:51:37.393 Crawling to find soname 'libsilkworm_capi.so'                                                                                                                                                                                         
2024-07-07 01:51:48.129 :: 2024-07-07 01:51:38.066 libsilkworm_capi.so with original path /root/go/pkg/mod/github.com/erigontech/silkworm-go@v0.14.0/lib/linux_x64/libsilkworm_capi.so found on '/root/go/pkg/mod/github.com/erigontech/silkworm-go@v0.14.0/lib/linux_x64/libsilkworm_capi.so' in base False                                                                                                                                                                                                                                                                       
2024-07-07 01:51:48.129 :: 2024-07-07 01:51:38.066 Crawling to find soname 'libstdc++.so.6'                                                                                                                                                                                              
2024-07-07 01:51:48.129 :: 2024-07-07 01:51:38.067 Extracting ELF attributes: '/snap/core22/current/lib/x86_64-linux-gnu/libstdc++.so.6'                                                                                                                                                 
2024-07-07 01:51:48.129 :: 2024-07-07 01:51:38.076 libstdc++.so.6 with original path /snap/core22/current/lib/x86_64-linux-gnu/libstdc++.so.6 found on '/snap/core22/current/lib/x86_64-linux-gnu/libstdc++.so.6' in base True

I can also see this from the Makefile which seems to extract that lib also as part of their build.

The file exists in the lxc container used for the build (but isn’t included in the snap):

snapcraft-erigon-on-amd64-for-amd64-14811307 # find . -name libsilkworm_capi.so                          
./go/pkg/mod/github.com/erigontech/silkworm-go@v0.14.0/lib/linux_x64/libsilkworm_capi.so

How can I mimic this behavior when building the go package, effectively including the lib in the snap?

Here is the snapcraft.yaml

name: erigon
title: Erigon
base: core22
adopt-info: erigon
summary: Erigon - Ethereum implementation, efficient and reliable.
license: GPL-3.0-with-GCC-exception
contact: info@dwellir.com
issues: https://github.com/dwellir-public/snap-erigon/issues
source-code: https://github.com/dwellir-public/snap-erigon
website: https://www.dwellir.com
grade: stable
confinement: strict

architectures:
  - amd64

description: |
  Erigon is an implementation of Ethereum (execution layer with embeddable consensus layer), on the efficiency
  frontier. Archive Node by default.

  Disclaimer: this software is currently a tech preview. We will do our best to keep it stable and make no breaking
  changes but we don't guarantee anything. Things can and will break.

  Important defaults: Erigon is an Archive Node by default (to remove history see: --prune flags in erigon --help). 
  We don't allow change this flag after first start.

  System Requirements
    - For an Archive node of Ethereum Mainnet we recommend >=3TB storage space: 1.8TB state (as of March 2022), 200GB temp
      files (can symlink or mount folder <datadir>/temp to another disk). Ethereum Mainnet Full node ( see --prune* flags): 400Gb (April 2022).
    - Goerli Full node (see --prune* flags): 189GB on Beta, 114GB on Alpha (April 2022).
    - Gnosis Chain Archive: 370GB (January 2023).
    - Polygon Mainnet Archive: 5TB. (April 2022). --prune.*.older 15768000: 5.1Tb (Sept 2023). Polygon Mumbai Archive: 1TB. (April 2022).
    - SSD or NVMe. Do not recommend HDD - on HDD Erigon will always stay N blocks behind chain tip, but not fall behind. Bear in mind that SSD performance deteriorates when close to capacity.
    - RAM: >= 16GB, 64-bit architecture.
    - Golang version >= 1.19; GCC 10+ or Clang; On Linux: kernel > v4

  Start the service with 'sudo snap start erigon'

  Check logs from the erigon service with 'snap logs erigon -f'

parts:
  erigon:
    plugin: go
    build-snaps: [go/latest/stable]
    source: https://github.com/ledgerwatch/erigon.git
    source-tag: v2.59.3
    override-pull: |
      craftctl default
      craftctl set version="2.59.3-$(git rev-parse --short HEAD)"
    organize:
      erigon: bin/erigon   

  wrappers:
    plugin: dump
    source-type: local
    source: .
    stage:
      - wrappers/

  utils:
    plugin: dump
    source: .
    stage:
      - utils/
apps:
  erigon-daemon:
    command: wrappers/start-erigon.bash
    daemon: simple
    install-mode: disable
    refresh-mode: endure
    restart-condition: never
    plugs:
      - network
      - network-bind
    environment:
      LC_ALL: C.UTF-8
      LANG: C.UTF-8
2

A quick look at the makefile leads me to erigon/Makefile at main · ledgerwatch/erigon · GitHub which takes care of copying that library into $(DIST), so you can replicate that line in override-build so it makes to a path within $CRAFT_PART_INSTALL or switch to the make plugin, but perhaps patch the Makefile to follow the conventional DESTDIR if it isn’t (I haven’t checked).

1 Like
3

Yes.

In the end I added a find for it and copied it into place (which is almost what is happening in the Makefile as you mentioned):

override-build: |
      # The Makefile contains bunch of compiler options. Don't even try reproduce it.
      go mod tidy
      make all
      make DIST=$SNAPCRAFT_PART_INSTALL install
      mkdir -p $SNAPCRAFT_PART_INSTALL/lib/

      # Rip out the libsilkworm_capi.so and ship it.
      find /root/go/pkg/mod/github.com/erigontech/ -name libsilkworm_capi.so -exec cp {} $SNAPCRAFT_PART_INSTALL/lib/ \;

    stage:
      - bin/erigon
      - lib/libsilkworm_capi.so

    organize:
      erigon: bin/erigon

… and added last in the snapcraft.yaml

# Make sure we can find libsilkworm_capi.so by setting LD_LIBRARY_PATH
environment:
  LD_LIBRARY_PATH: "$SNAP/lib:$LD_LIBRARY_PATH"

This seems to have resolved it.

1 Like