Spread tests fail on lack of /dev/ram0

snapd
1

I just noticed it here: https://travis-ci.org/snapcore/snapd/jobs/216691388

There are several errors there but if you go to linode:ubuntu-core-16-64:tests/main/snap-auto-import-asserts you can see the following failure:

-----
+ [[ ubuntu-core-16-64 != ubuntu-14.04-* ]]
+ journalctl --rotate
+ sleep .1
+ journalctl --vacuum-time=1ms
Deleted archived journal /run/log/journal/418ebfa33c584790b0af46dc0c96adfc/system@9246f0e749804a6d9b8aded033079b5f-0000000000000ba6-00054beffb5787e3.journal (2.5M).
Vacuuming done, freed 2.5M of archived journals on disk.
+ dmesg -c
+ /home/gopath/src/github.com/snapcore/snapd/tests/lib/reset.sh --reuse-core
+ [[ ubuntu-core-16-64 == ubuntu-core-16-* ]]
+ reset_all_snap --reuse-core
+ . /home/gopath/src/github.com/snapcore/snapd/tests/lib/names.sh
+++ sed -n 's/^\(pc\|pi[23]\|dragonboard\) .*/\1/p'
+++ snap list
++ gadget_name=pc
++ kernel_name=pc-kernel
++ '[' pc-kernel = pi3-kernel ']'
+ for snap in '/snap/*'
+ snap=bin
+ case "$snap" in
+ for snap in '/snap/*'
+ snap=core
+ case "$snap" in
+ for snap in '/snap/*'
+ snap=pc
+ case "$snap" in
+ for snap in '/snap/*'
+ snap=pc-kernel
+ case "$snap" in
+ systemctl stop snapd.service snapd.socket
+ rm -rf /var/lib/snapd/apparmor /var/lib/snapd/assertions /var/lib/snapd/desktop /var/lib/snapd/device /var/lib/snapd/hostfs /var/lib/snapd/mount /var/lib/snapd/seccomp /var/lib/snapd/seed /var/lib/snapd/snaps /var/lib/snapd/state.json
++ cd /
++ tar xzf /home/gopath/src/github.com/snapcore/snapd/snapd-state.tar.gz
+ rm -rf /root/.snap
+ '[' --reuse-core '!=' --keep-stopped ']'
+ systemctl start snapd.service snapd.socket
+ '[' production = staging ']'
+ . /home/gopath/src/github.com/snapcore/snapd/tests/lib/prepare.sh
++ set -eux
++ . /home/gopath/src/github.com/snapcore/snapd/tests/lib/apt.sh
++ . /home/gopath/src/github.com/snapcore/snapd/tests/lib/snaps.sh
+ [[ ubuntu-core-16-64 != ubuntu-core-16-* ]]
+ '[' true = false ']'
+ echo 'Ensure the testrootorg-store.account-key is not already added'
Ensure the testrootorg-store.account-key is not already added
++ wc -l
++ grep 'name: test-store'
++ snap known account-key
+ output=0
+ '[' 0 '!=' 0 ']'
+ echo 'Create a ramdisk with the testrootorg-store.account-key assertion'
Create a ramdisk with the testrootorg-store.account-key assertion
+ mkfs.vfat /dev/ram0
/dev/ram0: No such file or directory
mkfs.fat 3.0.28 (2015-05-16)
-----
2

We are using by default the kernel snap from edge in snapd’s spread execution for PRs, there has been a promotion recently that seems to be lacking the ram disks, this is the spread-cron execution after the promotion was detected https://travis-ci.org/snapcore/spread-cron/builds/216684131#L602

3

What should we do while this issue is present? Should I just ignore such failures?

4

This will affect all PRs, we could try to get the kernel snap at edge fixed or use a different channel, not sure what would be better or if there are more options, @niemeyer @mvo ?

5

note that this was me, i go over the beta and candidate channel for all kernels roughly once a month and release newer versions into edge from beta or candidate (the kernel team does not upload to edge at all, only to beta and candidate) to make sure edge stays up top date.

this means that the same kernel is already in candidate or beta …
if this caused issues, i can immediately go back a version in edge.

6

@ogra yes please :slight_smile: the previous edge version was fine

7

note though that your tests on beta and/or candidate will run into the same issue, technically i “backported” the snap from there.

are the tests running on i386 and amd64 or only amd64

8

Can we stop using edge in that case? I think it is fine to just use beta for all CI until the kernel team decides to offer nightly builds in edge.

9

@ogra @zyga-snapd @fgimenez have we spoken with the kernel people about this? is that an expected change we need to adapt to, or a regressions they will fix?

10

the point is that we also need to be able to test initrd changes and whatnot, for this we keep edge available for general development beyond the kernel team. the problem is really that the kernel team does not release into edge alongside with candidate here.

11

we only have amd64 based ubuntu-core systems on linode, so only amd64 channels are watched atm

12

I sent an email to brad figg when the kernel with the issues was promoted to beta the first time, @JamieBennett and @mvo were cc’ed, we haven’t received any response afaict.

13

ok, i rolled back to rev 55 on amd64 now … someone should file a bug though :wink:

14

+1, the default value is set here https://github.com/snapcore/snapd/blob/master/spread.yaml#L28, but we should wait until the snap at beta is fixed.

We are watching all the channels anyway, as soon as any change is detected on a channel the suite is executed using the kernel snap from that channel.

15

i think actually some mis-communication … while we need to be able to upload kernel snaps to edge for non kernel change testing (initrd changes, firmware additions etc), we still want them to upload to edge at the same time they push to candidate so the edge version stays up to date.

16

I think it would also help if the kernel snap was on github. I tried to make this happen and asked Leann about it but I think the conversation didn’t lead anywhere in the end. My point is that the kind of initrd/firmware experiments should be simply pull requests to that snap.

17

these “experiments” (i’d call that normal development) are maintained in packages (and pulled in from the image PPA during the kernel snap build) … with the addition of initramfs-tools-ubuntu-core to https://github.com/snapcore/core-build we’ll automatically have reviews for the initrd changes anyway.

i doubt the kernel team will want to move away from kernel.ubuntu.com for the build scripts … (and they are not really relevant for things like initrd changes after all, it is just a Makefile and snapcraft.yaml)
perhaps we could just have a readonly clone of the kernel.ubuntu.com tree on GH to make it easier for people to look at the code ? is that supported by GH ?