Pre-seeded cloud-init config is ignored for core18 image

device
1

I am creating a custom core18-amd64 image using ubuntu-image tool and ubuntu-core-18-amd64.model-assertion.

ubuntu-image snap --snap docker --snap avahi --snap myprivatesnap_amd64.snap --cloud-init cloud-config.yaml ubuntu-core-18-amd64.model-assertion

cloud-config.yaml is very simple. It just runs a few snap commands to configure myprivatesnap:

#cloud-config
snap:
  commands:
    - snap connect myprivatesnap:avahi-control avahi
    - snap connect myprivatesnap:docker-daemon docker
    - snap connect myprivatesnap:docker-executables docker
    - snap connect myprivatesnap:hardware-observe
    - snap set myprivatesnap foo=bar

I have verified that the cloud-config.yaml is correctly written to /var/lib/cloud/seed/nocloud-net in the resulting image. However after cloud-init runs, /var/lib/cloud/instance/user-data.txt is empty. From /var/log/cloud-init.log, it seems that /var/lib/cloud/seed/nocloud-net/user-data is read but somehow get dropped.

I also see the following errors in /var/log/cloud-init.log:
OSError: [Errno 30] Read-only file system: ā€˜/etc/pollinateā€™
OSError: [Errno 30] Read-only file system: ā€˜/etc/localtimeā€™

/var/lib/cloud/data/status.json:

{
 "v1": {
  "datasource": "DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net,/dev/sr0][dsmode=net]",
  "init": {
   "errors": [
    "('write-files', OSError(30, 'Read-only file system'))"
   ],
   "finished": 1588981155.247145,
   "start": 1588981154.733196
  },
  "init-local": {
   "errors": [],
   "finished": 1588981152.377084,
   "start": 1588981151.9045477
  },
  "modules-config": {
   "errors": [
    "('timezone', OSError(30, 'Read-only file system'))"
   ],
   "finished": 1588981156.988874,
   "start": 1588981156.7807033
  },
  "modules-final": {
   "errors": [],
   "finished": 1588981157.4702873,
   "start": 1588981157.3532157
  },
  "modules-init": {
   "errors": [],
   "finished": null,
   "start": null
  },
  "stage": null
 }
}

However when I pass the cloud-config file when launching the image via multipass, it does get picked up and all cloud-init commands are correctly executed:

multipass launch --cloud-init cloud-config.yaml file:///pc.img

I could use some help in troubleshooting why the pre-seeded cloud-init config is ignored while the same file gets correctly picked up by multipass. Thanks!

2

Hi @alok,

Multipass builds an .iso as the data source, that must be what makes it different. The OSErrors shouldnā€™t make the whole thing break. Iā€™d suspect your config isnā€™t picked up at all.

Youā€™ll have to retrace the cloud-init.log, especially wrt where it picks the data up.

1 Like
3

Thanks for the hint Michal.

I looked at the cloud-init logs again and noticed that DataSourceNoCloud is reading from two locations - seed directory and an iso (apparently written by multipass).

/run/cloud-init/result.json:

{
 "v1": {
  "datasource": "DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net,/dev/sr0][dsmode=net]",
  "errors": [
   "('write-files', OSError(30, 'Read-only file system'))",
   "('timezone', OSError(30, 'Read-only file system'))"
  ]
 }
}

Do you know if user-data from multiple sources is ā€œmergedā€ or ā€œoverwrittenā€? I could not find this information in cloud-init documentation. The behavior I am seeing is ā€œoverwriteā€.

More relevant logs from cloud-init.log:

util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/user-data (quiet=False)
util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/meta-data (quiet=False)
util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/vendor-data (quiet=False)
util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud/network-config (quiet=False)
util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/user-data (quiet=False)
util.py[DEBUG]: Read 64 bytes from /var/lib/cloud/seed/nocloud-net/user-data
util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/meta-data (quiet=False)
util.py[DEBUG]: Read 28 bytes from /var/lib/cloud/seed/nocloud-net/meta-data
util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/vendor-data (quiet=False)
util.py[DEBUG]: Reading from /var/lib/cloud/seed/nocloud-net/network-config (quiet=False)
DataSourceNoCloud.py[DEBUG]: Using seeded data from /var/lib/cloud/seed/nocloud-net
util.py[DEBUG]: Attempting to load yaml from string of length 28 with allowed root types (<class 'dict'>,)

DataSourceNoCloud.py[DEBUG]: Attempting to use data from /dev/sr0
util.py[DEBUG]: Running command ['mount', '-o', 'ro', '-t', 'auto', '/dev/sr0', '/run/cloud-init/tmp/tmpm0ydfqvf'] with allowed return codes [0] (shell=False, capture=True)
Reading from /run/cloud-init/tmp/tmpm0ydfqvf//user-data (quiet=False)
util.py[DEBUG]: Read 17 bytes from /run/cloud-init/tmp/tmpm0ydfqvf//user-data
util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmpm0ydfqvf//meta-data (quiet=False)
util.py[DEBUG]: Read 63 bytes from /run/cloud-init/tmp/tmpm0ydfqvf//meta-data
util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmpm0ydfqvf//vendor-data (quiet=False)
util.py[DEBUG]: Read 877 bytes from /run/cloud-init/tmp/tmpm0ydfqvf//vendor-data
util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmpm0ydfqvf//network-config (quiet=False)
util.py[DEBUG]: Running command ['umount', '/run/cloud-init/tmp/tmpm0ydfqvf'] with allowed return codes [0] (shell=False, capture=True)
util.py[DEBUG]: Attempting to load yaml from string of length 63 with allowed root types (<class 'dict'>,)
DataSourceNoCloud.py[DEBUG]: Using data from /dev/sr0

atomic_helper.py[DEBUG]: Atomically writing to file /run/cloud-init/instance-data.json (via temporary file /run/cloud-init/tmp4lq18nbu) - w: [644] 755 bytes/chars
atomic_helper.py[DEBUG]: Atomically writing to file /run/cloud-init/instance-data-sensitive.json (via temporary file /run/cloud-init/tmpitaky9sw) - w: [600] 755 bytes/chars
handlers.py[DEBUG]: finish: init-local/search-NoCloud: SUCCESS: found local data from DataSourceNoCloud
stages.py[INFO]: Loaded datasource DataSourceNoCloud - DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net,/dev/sr0][dsmode=net]
4

Hi @alok,

nocloud-net is a ā€œvirtualā€ datasource that would require you to pass in details about the source (usually on kernel command line). Since thatā€™s not happening, this does:

DataSourceNoCloud.py[DEBUG]: Using data from /dev/sr0

So itā€™s only the ISO file that is being read in - and as you say, everything worked in this case.

It would be interesting to see the cloud-init logs from when it fails. The write-files and timezone errors are due to how Core is set up, and cloud-init apparently doesnā€™t deal well with that. It may be worth filing a bug about the timezone module at least - write-files youā€™ll need to modify in a way that it writes to a writable location.

Hope this helps.