Long time for first boot

Hi there,
we are experiencing a rather long first boot time for our board using ubuntu core 16: we are experiencing value between 7 and 15 minutes.
I mean for boot time the time between “the power on” action and the presence of the console-conf frontend.
We build our image with two our other snap that register themselves to our cloud.
The procedure of registration to our cloud takes some seconds: we measure it using state change of some led on the board.
It seem the system take a long time to boot up all the snap, then when the snaps are running the responsiveness of the board is good.

Could someone help us to understand if that long time is expected? If not, could someone share some advice for debug the source of this issue?

Thanks in advance

Nicolino

PS: we are using a brand store and our board is a Allwinner H3 running a 4.9 kernel patched following the @ogra reccomendations.

this long time is definitely not expected, while there are snaps being copied in place on the very first boot, this should not take much time. i would consider anything above 1-2minutes for the very first boot a bug.

i would suggest building an image without any additional snaps to rule out any issues with the preinstalled app snaps. also note that there is some gpg key generation happening on first boot. setting “rng_core.default_quality=700” on your default kernel commandline will significantly speed up that step.

beyond this … logs would be helpful indeed :wink:

2 Likes

thanks @ogra.
I found into the log before application of your hint and with 2 snaps installed on images:

Jun 12 07:13:22 localhost /usr/lib/snapd/snapd[1242]: taskrunner.go:367: DEBUG: Running task 53 on Do: Generate device key

and

Jun 12 07:21:10 localhost /usr/lib/snapd/snapd[1242]: taskrunner.go:367: DEBUG: Running task 1 on Do: Mark system seeded

After the application of rng_core.default_quality=700 we don’t notice any changes on timing of device key generation.
We are sure that we are setting this parameters because the command cat /proc/cmdline show:

root=/dev/disk/by-label/writable net.ifnames=0 init=/lib/systemd/systemd ro panic=-1 fixrtc rng_core.default_quality=700 snap_core=core_1690.snap snap_kernel=domotz-platform-kernel_1.snap rootfstype=ext4 rootwait ath9k_htc.domotz_ath_hwaddr=0:e:3f:23:22:13

During the test without our snaps in the image we have the followings:

Jun 12 13:16:31 localhost /usr/lib/snapd/snapd[1302]: taskrunner.go:367: DEBUG: Running task 33 on Do: Generate device key 

and

Jun 12 13:16:30 localhost /usr/lib/snapd/snapd[1302]: taskrunner.go:367: DEBUG: Running task 1 on Do: Mark system seeded

The following is the log for the last attempt:

Nicolino

ah, thanks for the syslog … this clearly shows that your key generation is fine, but it also shows that your network device seems to drop out in a loop.

Jun 12 13:16:30 localhost kernel: [ 220.864397] usb 4-1: ath9k_htc: Firmware - ath9k_htc/htc_9271-1.4.0.fw download failed

looks very suspicious … you might need to make sure your kernel snap ships the required firmware in the right place …

thanks, @ogra.
Our kernel snapcraft.yaml is the following apart from the jinjaed part :

name: {{ board }}-kernel
confinement: strict
grade: stable
version: 4.9.5
summary: "A {{ board }} kernel built from source"
description: {{ package_description }}
type: kernel
architectures: [ armhf ]

parts:
    kernel:
        plugin: kernel
        source: {{ kernel_repository_url }}
{% if kernel_source_branch is defined %}
        source-branch: {{ kernel_source_branch }}
{% endif %}
{% if kernel_source_tag is defined %}
        source-tag: {{ kernel_source_tag }}
{% endif %}
{% if kernel_source_commit is defined %}
        source-commit: {{ kernel_source_commit }}
{% endif %}
        source-type: git
        source-depth: 15
        kdefconfig: [ {{ kernel_defconfig }}_defconfig ]
        kconfigs:
            - CONFIG_LOCALVERSION="-{{ board }}"
            - CONFIG_DEBUG_INFO=n
            - CONFIG_SQUASHFS=m
            - CONFIG_DEVPTS_MULTIPLE_INSTANCES=y
            - CONFIG_BT_BCM=m
            - CONFIG_MAC80211_LEDS=y
            - CONFIG_RFKILL_LEDS=y
            - CONFIG_LEDS_TRIGGERS=y
            - CONFIG_RTL8189FS=n
            - CONFIG_REALTEK_PHY=n
            - CONFIG_SUN8I_THS=m
            - CONFIG_THERMAL=m
            - CONFIG_BINFMT_MISC=y
            - CONFIG_CPUFREQ_DT=m
            - CONFIG_ATH9K_HW=m
            - CONFIG_ATH9K_COMMON=m
            - CONFIG_ATH9K_HTC=m
            - CONFIG_ATH9K=m
            - CONFIG_MAC80211=m
            - CONFIG_FIRMWARE_IN_KERNEL=n

        kernel-initrd-modules:
            - squashfs

        kernel-image-target: zImage
        kernel-device-trees:
            - {{ dts }}
        build-packages: [bc, kmod, cpio]
        after: [firmware]

    firmware:
        plugin: tar-content
        source: firmware.tar
        destination: firmware 

Where is the problem?

Furthermore, our wireless network card is affected from the behaviour of netplan together to other models: are there other process that reconfigure the network card during the first boot?

Thanks for your help.

cheers,
Nicolino

what is the content of firmware.tar ? if it has any extra subdirs in it the driver might not find what it is looking for …
http://paste.ubuntu.com/24841879/ is a dump of the firmware dir from my linux-generic-bbb package (which i use in various board images)

netplan is usually configured via console-conf, but before console-conf starts there is a generic dhcp client starting, so all wired interfaces (seems you have an eth0 device in your syslog) should come up with whatever the DHCP server provides as default.

for wireless pre-configuration you might need to use the network-manager snap, perhaps @morphis has some hints about that setup.