Unable to install microk8s on RaspberryPi due to configuration hook timeout

snap
1

I am attempting to install the microk8s snap on a Raspberry Pi 3 B+, using these instructions: https://ubuntu.com/tutorials/how-to-kubernetes-cluster-on-raspberry-pi#1-overview

I freshly imaged a card using Raspberry Pi Imager v1.3 and the Ubuntu 20.04.1 LTS 64-bit ARM image.

However, when I run the command:

sudo snap install microk8s --classic

The install times out during the configure hook:

ubuntu@ubuntu:/var/log$ sudo snap install microk8s --classic
2020-09-09T18:37:53Z INFO Waiting for automatic snapd restart...
error: cannot perform the following tasks:
- Run configure hook of "microk8s" snap if present (run hook "configure":
-----
++ '[' -f /var/snap/microk8s/1670/certs/server.key ']'
++ for key in serviceaccount.key ca.key server.key front-proxy-ca.key front-proxy-client.key
++ '[' -f /var/snap/microk8s/1670/certs/front-proxy-ca.key ']'
++ for key in serviceaccount.key ca.key server.key front-proxy-ca.key front-proxy-client.key
++ '[' -f /var/snap/microk8s/1670/certs/front-proxy-client.key ']'
++ '[' -f /var/snap/microk8s/1670/certs/ca.crt ']'
++ '[' -f /var/snap/microk8s/1670/certs/front-proxy-ca.crt ']'
++ render_csr_conf
+++ get_ips
++++ /snap/microk8s/1670/bin/hostname -I
+++ local 'IP_ADDR=192.168.1.206 '
+++ [[ -z 192.168.1.206  ]]
+++ /snap/microk8s/1670/sbin/ifconfig cni0
+++ echo '192.168.1.206 '
++ local 'IP_ADDRESSES=192.168.1.206 '
++ cp /var/snap/microk8s/1670/certs/csr.conf.template /var/snap/microk8s/1670/certs/csr.conf.rendered
++ '[' '192.168.1.206 ' == 127.0.0.1 ']'
++ '[' '192.168.1.206 ' == none ']'
++ local ips= sep=
++ local -i i=3
+++ echo '192.168.1.206 '
++ for IP_ADDR in $(echo "$IP_ADDRESSES")
++ ips+='IP.3 = 192.168.1.206'
++ sep='\n'
++ /snap/microk8s/1670/bin/sed -i 's/#MOREIPS/IP.3 = 192.168.1.206/g' /var/snap/microk8s/1670/certs/csr.conf.rendered
++ '[' -f /var/snap/microk8s/1670/certs/csr.conf ']'
++ local force
++ /snap/microk8s/1670/usr/bin/cmp -s /var/snap/microk8s/1670/certs/csr.conf.rendered /var/snap/microk8s/1670/certs/csr.conf
++ force=false
++ false
++ '[' '!' -f /var/snap/microk8s/1670/certs/front-proxy-client.crt ']'
+++ /snap/microk8s/1670/usr/bin/openssl x509 -noout -issuer
++ '[' 'issuer= /CN=front-proxy-ca' == 'issuer=CN = 127.0.0.1' ']'
++ echo 0
+ '[' 0 == 1 ']'
+ '[' -e /var/snap/microk8s/1670/args/containerd-template.toml ']'
+ grep -e 'stream_server_address = ""' /var/snap/microk8s/1670/args/containerd-template.toml
+ grep -e '\-\-allow-privileged' /var/snap/microk8s/1670/args/kubelet
+ '[' -f /root/snap/microk8s/common/istio-auth.lock ']'
+ '[' -f /root/snap/microk8s/common/istio-auth.lock ']'
+ need_api_restart=false
+ '[' -f /var/snap/microk8s/1670/credentials/kubelet.config ']'
+ '[' -f /var/snap/microk8s/1670/credentials/proxy.config ']'
+ '[' -f /var/snap/microk8s/1670/credentials/scheduler.config ']'
+ '[' -f /var/snap/microk8s/1670/credentials/controller.config ']'
+ for dir in "${SNAP_DATA}/credentials/ ${SNAP_DATA}/certs/ ${SNAP_DATA}/args/ ${SNAP_DATA}/var/lock"
+ chmod -R ug+rwX /var/snap/microk8s/1670/credentials/ /var/snap/microk8s/1670/certs/ /var/snap/microk8s/1670/args/ /var/snap/microk8s/1670/var/lock
+ chmod -R o-rwX /var/snap/microk8s/1670/credentials/ /var/snap/microk8s/1670/certs/ /var/snap/microk8s/1670/args/ /var/snap/microk8s/1670/var/lock
+ getent group microk8s
+ getent group microk8s
+ chgrp microk8s -R /var/snap/microk8s/1670/credentials/ /var/snap/microk8s/1670/certs/ /var/snap/microk8s/1670/args/ /var/snap/microk8s/1670/var/lock/ /var/snap/microk8s/1670/var/kubernetes/backend/
+ '[' -e /var/snap/microk8s/1670/opt/cni/bin/flanneld ']'
+ '[' -f /var/snap/microk8s/1670/args/flanneld ']'
+ grep -e etcd.socket:2379 /var/snap/microk8s/1670/args/etcd
+ grep -e basic-auth-file /var/snap/microk8s/1670/args/kube-apiserver
+ grep '\-\-enable\-v2' /var/snap/microk8s/1670/args/etcd
--enable-v2=true
+ false
+ '[' -L /var/snap/microk8s/1670/bin/cilium ']'
+ '[' -e /var/snap/microk8s/1670/var/lock/clustered.lock ']'
+ '[' -e /var/snap/microk8s/1670/args/flannel-template.conflist ']'
+ grep -e cniVersion /var/snap/microk8s/1670/args/flannel-template.conflist
    "cniVersion": "0.3.1",
+ '[' '!' -f /var/snap/microk8s/1670/args/cluster-agent ']'
+ grep -e '\-\-timeout' /var/snap/microk8s/1670/args/cluster-agent
--timeout 240
+ mkdir -p /var/snap/microk8s/1670/juju/share/juju /var/snap/microk8s/1670/juju-home
+ chmod -R ug+rwX /var/snap/microk8s/1670/juju /var/snap/microk8s/1670/juju-home
+ chmod -R o-rwX /var/snap/microk8s/1670/juju /var/snap/microk8s/1670/juju-home
+ getent group microk8s
+ chgrp microk8s -R /var/snap/microk8s/1670/juju /var/snap/microk8s/1670/juju-home
+ grep -e '\-\-ip-masq' /var/snap/microk8s/1670/args/flanneld
--ip-masq=true
+ grep -e '\-\-cluster-cidr=10.152.183.0/24' /var/snap/microk8s/1670/args/kube-proxy
+ '[' -e /var/snap/microk8s/1670/args/cni-network/cni.yaml ']'
+ '[' -e /var/snap/microk8s/1670/var/lock/ha-cluster ']'
+ echo 'Setting up the CNI'
Setting up the CNI
++ date +%s
+ start_timer=1599676772
+ timeout=120
+ KUBECTL='/snap/microk8s/1670/kubectl --kubeconfig=/var/snap/microk8s/1670/credentials/client.config'
+ sleep 5
++ date +%s
+ now=1599676791
+ [[ 1599676791 > 1599676892 ]]
+ sleep 5
++ date +%s
+ now=1599676797
+ [[ 1599676797 > 1599676892 ]]
+ sleep 5
++ date +%s
+ now=1599676803
+ [[ 1599676803 > 1599676892 ]]
+ sleep 5
++ date +%s
+ now=1599676886
+ [[ 1599676886 > 1599676892 ]]

<exceeded maximum runtime of 5m0s>
-----)

Other system info:

ubuntu@ubuntu:/var/log$ uname -a
Linux ubuntu 5.4.0-1018-raspi #20-Ubuntu SMP Sun Sep 6 05:11:16 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

ubuntu@ubuntu:/var/log$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.1 LTS
Release:        20.04
Codename:       focal

I have attempted to fully reinstall snapd, with no change.

I have been unable to find any way to increase the timeout, or any other instance of this particular issue. Any ideas on how to avoid this timeout, or just increase the time allowed for the configure hook?

Thanks!

2

Looks like this might be an issue with the latest branch (1.19). This install works as expected:

sudo snap install microk8s --classic --channel=1.18/stable
3

Having the same issue on a freshly imaged Pi 3B and 3B+ with version 1.19/stable. I think you may be right. Channel 1.18/stable worked great.

System Info (both boxes are identical except time):

ubuntu@ubuntu:~$ uname -a
Linux ubuntu 5.4.0-1018-raspi #20-Ubuntu SMP Sun Sep 6 05:11:16 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@ubuntu:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.1 LTS
Release:        20.04
Codename:       focal

Thanks for the tip @pokkiff.

4

After installing 1.18 successfully, I was able to then upgrade to 1.19 successfully:

ubuntu@ubuntu:~$ sudo snap refresh microk8s --classic --channel=1.19/stable
microk8s (1.19/stable) v1.19.0 from Canonical✓ refreshed
5

My cluster uses a mix of Pi 3s and 4s.
I have found that 1.19 will install and work on the 4s, but not the 3s.
For the 3s I’ve had to use 1.18.

6

Success!

I was able to get 1.19 to upgrade from 1.18, but it was useless since hardly any of the microk8s commands worked.

Okay, so it looks like 1.19 may need more memory than the RPi3 can provide. I followed the instructions here https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-20-04 to create a 3GB swap file on my SD card (I know it is not a good idea to keep swap on SD cards, this is a test) and and enabled it. For giggles, I also added it to my fstab so it would survive a reboot. 1.19 installed fine. I haven’t had time to test it yet, but it looks like 1.19 just needs too much RAM to be successful on the Raspberry Pi 3s.