We’ve recently had a bug filed on GitHub (https://github.com/ubuntu/microk8s/issues/2336) where it seems that, on RHEL 8 based OSes, our ARM64 build does not work. I’ve tested this build on Ubuntu 20.04, which works. I’ve also confirmed that it does not work on RHEL 8 based OSes, namely RHEL 8 itself.
It seems some weird library issue, perhaps where Ubuntu is allowing system libraries to be used. If I slowly copy RHEL binaries into the snap, I can get these to work, but it fails on the next binary call:
E.g. after replacing cp with cp from the OS.
[ec2-user@ip-172-31-28-121 ~]$ sudo snap install ./mk8s.snap --dangerous --classic
error: cannot perform the following tasks:
- Run install hook of "microk8s" snap if present (run hook "install":
-----
++ /snap/microk8s/x1/bin/uname -m
++ xargs echo -n
+ ARCH=aarch64
+ export LD_LIBRARY_PATH=/snap/microk8s/x1/lib:/snap/microk8s/x1/usr/lib:/snap/microk8s/x1/lib/aarch64-linux-gnu:/snap/microk8s/x1/usr/lib/aarch64-linux-gnu
+ LD_LIBRARY_PATH=/snap/microk8s/x1/lib:/snap/microk8s/x1/usr/lib:/snap/microk8s/x1/lib/aarch64-linux-gnu:/snap/microk8s/x1/usr/lib/aarch64-linux-gnu
+ export PATH=/snap/microk8s/x1/usr/sbin:/snap/microk8s/x1/usr/bin:/snap/microk8s/x1/sbin:/snap/microk8s/x1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/bin:/usr/local/bin
+ PATH=/snap/microk8s/x1/usr/sbin:/snap/microk8s/x1/usr/bin:/snap/microk8s/x1/sbin:/snap/microk8s/x1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/bin:/usr/local/bin
+ export OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ source /snap/microk8s/x1/actions/common/utils.sh
+ /snap/microk8s/x1/bin/cp -r /snap/microk8s/x1/default-args /var/snap/microk8s/x1/args
+ '[' -e /var/lib/kubelet ']'
+ ln -s /var/snap/microk8s/common/var/lib/kubelet /var/lib/kubelet
ln: failed to create symbolic link '/var/lib/kubelet': File exists
+ echo '`/var/lib/kubelet` already exists. Will not be linking to /var/snap/microk8s/common'
`/var/lib/kubelet` already exists. Will not be linking to /var/snap/microk8s/common
+ mkdir /var/snap/microk8s/x1/certs
+ /snap/microk8s/x1/bin/cp /snap/microk8s/x1/certs/csr.conf.template /var/snap/microk8s/x1/certs/csr.conf.template
+ produce_certs
+ export OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ for key in serviceaccount.key ca.key server.key front-proxy-ca.key front-proxy-client.key
+ '[' -f /var/snap/microk8s/x1/certs/serviceaccount.key ']'
+ /snap/microk8s/x1/usr/bin/openssl genrsa -out /var/snap/microk8s/x1/certs/serviceaccount.key 2048
/snap/microk8s/x1/actions/common/utils.sh: line 396: 67584 Segmentation fault ${SNAP}/usr/bin/openssl genrsa -out ${SNAP_DATA}/certs/$key 2048
Then, after replacing openssl:
[ec2-user@ip-172-31-28-121 ~]$ sudo snap install ./mk8s.snap --dangerous --classic
error: cannot perform the following tasks:
- Run install hook of "microk8s" snap if present (run hook "install":
-----
++ /snap/microk8s/x1/bin/uname -m
++ xargs echo -n
+ ARCH=aarch64
+ export LD_LIBRARY_PATH=/snap/microk8s/x1/lib:/snap/microk8s/x1/usr/lib:/snap/microk8s/x1/lib/aarch64-linux-gnu:/snap/microk8s/x1/usr/lib/aarch64-linux-gnu
+ LD_LIBRARY_PATH=/snap/microk8s/x1/lib:/snap/microk8s/x1/usr/lib:/snap/microk8s/x1/lib/aarch64-linux-gnu:/snap/microk8s/x1/usr/lib/aarch64-linux-gnu
+ export PATH=/snap/microk8s/x1/usr/sbin:/snap/microk8s/x1/usr/bin:/snap/microk8s/x1/sbin:/snap/microk8s/x1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/bin:/usr/local/bin
+ PATH=/snap/microk8s/x1/usr/sbin:/snap/microk8s/x1/usr/bin:/snap/microk8s/x1/sbin:/snap/microk8s/x1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/bin:/usr/local/bin
+ export OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ source /snap/microk8s/x1/actions/common/utils.sh
+ /snap/microk8s/x1/bin/cp -r /snap/microk8s/x1/default-args /var/snap/microk8s/x1/args
+ '[' -e /var/lib/kubelet ']'
+ ln -s /var/snap/microk8s/common/var/lib/kubelet /var/lib/kubelet
ln: failed to create symbolic link '/var/lib/kubelet': File exists
+ echo '`/var/lib/kubelet` already exists. Will not be linking to /var/snap/microk8s/common'
`/var/lib/kubelet` already exists. Will not be linking to /var/snap/microk8s/common
+ mkdir /var/snap/microk8s/x1/certs
+ /snap/microk8s/x1/bin/cp /snap/microk8s/x1/certs/csr.conf.template /var/snap/microk8s/x1/certs/csr.conf.template
+ produce_certs
+ export OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ for key in serviceaccount.key ca.key server.key front-proxy-ca.key front-proxy-client.key
+ '[' -f /var/snap/microk8s/x1/certs/serviceaccount.key ']'
+ /snap/microk8s/x1/usr/bin/openssl genrsa -out /var/snap/microk8s/x1/certs/serviceaccount.key 2048
/snap/microk8s/x1/usr/bin/openssl: error while loading shared libraries: libssl.so.1.1: ELF load command alignment not page-aligned
…you get the picture.
This same snap (well the x86_64 build of the same snap) works fine on Ubuntu and RHEL 8 x86_64. The issue only presents itself on ARM.