Need help to track down network errors in snapcraft

joew · May 7, 2024, 12:10pm

Hello folks,

I need some assistance to track down a problem when using snapcraft to build subiquity.

Those commands worked fine a while ago when I was building subiquity in order to remaster ubuntu-22.04. Now, when trying to remaster ubuntu-24.04, I get

  craft-providers error: A network related operation failed in a context of no network access.
  Verify that the environment has internet connectivity; see https://craft-providers.readthedocs.io/ for further reference.
  Full execution log: '/home/jw/.local/state/snapcraft/log/snapcraft-20240507-133748.717420.log'

Unfortunately, there’s no indication which command failed and what exact the error was. Transcript is attached below.

Any ideas what might be broken here?

$ git clone https://github.com/canonical/subiquity.git
Cloning into 'subiquity'...
remote: Enumerating objects: 43854, done.
remote: Counting objects: 100% (1635/1635), done.
remote: Compressing objects: 100% (768/768), done.
remote: Total 43854 (delta 1041), reused 1356 (delta 858), pack-reused 42219
Receiving objects: 100% (43854/43854), 12.60 MiB | 2.39 MiB/s, done.
Resolving deltas: 100% (33702/33702), done.
$ ( cd subiquity && make install_deps )
sudo apt update && \
sudo apt-get install -y build-essential cloud-init curl dctrl-tools efibootmgr fuseiso gettext gir1.2-umockdev-1.0 git isolinux jq language-selector-common libnl-3-dev libnl-genl-3-dev libnl-route-3-dev libsystemd-dev lsb-release os-prober pandoc pkg-config python3-aiohttp python3-aioresponses python3-apport python3-async-timeout python3-attr python3-bson python3-coverage python3-debian python3-dev python3-distro-info python3-distutils-extra python3-flake8 python3-gi python3-jsonschema python3-more-itertools python3-mypy python3-nose python3-parameterized python3-pip python3-pyflakes python3-pyroute2 python3-pytest python3-pytest-xdist python3-pyudev python3-requests python3-requests-mock python3-requests-unixsocket python3-setuptools python3-systemd python3-urwid python3-wheel python3-yaml ssh-import-id ubuntu-drivers-common umockdev xorriso
[ ... ]
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
./scripts/update-part.py curtin
running ('git', 'clone', 'https://git.launchpad.net/curtin', 'curtin')
Cloning into 'curtin'...
remote: Enumerating objects: 27137, done.
remote: Counting objects: 100% (27137/27137), done.
remote: Compressing objects: 100% (8668/8668), done.
remote: Total 27137 (delta 19813), reused 25046 (delta 18295)
Receiving objects: 100% (27137/27137), 3.81 MiB | 4.61 MiB/s, done.
Resolving deltas: 100% (19813/19813), done.
./scripts/update-part.py probert
running ('git', 'clone', 'https://github.com/canonical/probert.git', 'probert')
Cloning into 'probert'...
remote: Enumerating objects: 1800, done.
remote: Counting objects: 100% (332/332), done.
remote: Compressing objects: 100% (141/141), done.
remote: Total 1800 (delta 212), reused 282 (delta 186), pack-reused 1468
Receiving objects: 100% (1800/1800), 431.18 KiB | 2.91 MiB/s, done.
Resolving deltas: 100% (1202/1202), done.
running ('git', '-c', 'advice.detachedHead=false', '-C', 'probert', 'checkout', 'f34ae3c3f942a9c479fe928911053a302e146251')
HEAD is now at f34ae3c Merge pull request #140 from ogayot/nvme-o-tcp
(cd probert && python3 setup.py build_ext -i);
running build_ext
[ ... ]
$ ( cd subiquity && git clone https://github.com/mwhudson/livefs-editor )
Cloning into 'livefs-editor'...
remote: Enumerating objects: 599, done.
remote: Counting objects: 100% (131/131), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 599 (delta 93), reused 95 (delta 78), pack-reused 468
Receiving objects: 100% (599/599), 127.01 KiB | 1.40 MiB/s, done.
Resolving deltas: 100% (373/373), done.
$ ( cd subiquity && snapcraft pull curtin )
craft-providers error: A network related operation failed in a context of no network access.
Verify that the environment has internet connectivity; see https://craft-providers.readthedocs.io/ for further reference.                     
Full execution log: '/home/jw/.local/state/snapcraft/log/snapcraft-20240507-133748.717420.log'
$

joew · May 7, 2024, 12:11pm

Here is the logfile created by snapcraft:

2024-05-07 13:37:48.718 Starting snapcraft, version 8.2.4
2024-05-07 13:37:48.718 Log verbosity level set to BRIEF
2024-05-07 13:37:48.727 Falling back from craft-application to snapcraft.
2024-05-07 13:37:48.728 lifecycle command: 'pull', arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=False, manifest_image_information=None, bind_ssh=False, build_for=None, http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, parts=['curtin'], shell=False, shell_after=False)
2024-05-07 13:37:48.728 command: pull, arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=False, manifest_image_information=None, bind_ssh=False, build_for=None, http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, parts=['curtin'], shell=False, shell_after=False)
2024-05-07 13:37:48.781 CPU count (from process affinity): 12
2024-05-07 13:37:48.781 Invalid SNAPCRAFT_MAX_PARALLEL_BUILD_COUNT ''
2024-05-07 13:37:48.782 Running on amd64 for amd64
2024-05-07 13:37:48.784 Checking build provider availability
2024-05-07 13:37:48.790 Retrieved snap config: {}
2024-05-07 13:37:48.790 Using default provider 'lxd' on linux system.
2024-05-07 13:37:48.959 Executing on host: lxc --project default profile show local:default
2024-05-07 13:37:49.056 Using hostname 'snapcraft-subiquity-on-amd64-for-amd64-93859022'
2024-05-07 13:37:49.056 Launching instance...
2024-05-07 13:37:49.056 Executing on host: lxc remote list --format=yaml
2024-05-07 13:37:49.147 Remote 'craft-com.ubuntu.cloud-buildd' already exists.
2024-05-07 13:37:49.147 Executing on host: lxc project list local: --format=yaml
2024-05-07 13:37:49.240 Set LXD instance name to 'snapcraft-subiquity-on-amd64-for-amd64-93859022'
2024-05-07 13:37:49.240 Checking for instance 'snapcraft-subiquity-on-amd64-for-amd64-93859022' in project 'snapcraft' in remote 'local'
2024-05-07 13:37:49.240 Executing on host: lxc --project snapcraft list local: --format=yaml
2024-05-07 13:37:49.334 Instance 'snapcraft-subiquity-on-amd64-for-amd64-93859022' does not exist.
2024-05-07 13:37:49.335 Set LXD instance name to 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861'
2024-05-07 13:37:49.335 Checking for base instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861' in project 'snapcraft' in remote 'local'
2024-05-07 13:37:49.335 Executing on host: lxc --project snapcraft list local: --format=yaml
2024-05-07 13:37:49.427 Checking validity of instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861'.
2024-05-07 13:37:49.427 Executing on host: lxc --project snapcraft info local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861
2024-05-07 13:37:49.514 Executing on host: lxc --project snapcraft info local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861
2024-05-07 13:37:49.599 Executing on host: lxc --project snapcraft config get local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.status
2024-05-07 13:37:49.690 Instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861' is not ready (State: RUNNING, Status: PREPARING)
2024-05-07 13:37:49.690 Executing on host: lxc --project snapcraft config get local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.pid
2024-05-07 13:37:49.774 Instance is not valid: Instance is not ready.
Instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861' is not ready and the process (pid 4085736) that created the instance is inactive.
2024-05-07 13:37:49.774 Base instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861' is not valid. Deleting base instance.
2024-05-07 13:37:49.774 Executing on host: lxc --project snapcraft delete local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 --force
2024-05-07 13:37:51.399 Creating new instance from remote
2024-05-07 13:37:51.399 Creating new base instance from remote
2024-05-07 13:37:51.399 Creating new base instance from image 'core22' from remote 'craft-com.ubuntu.cloud-buildd'
2024-05-07 13:37:51.399 Executing on host: lxc --project snapcraft info local:
2024-05-07 13:37:51.498 Executing on host: lxc --project snapcraft launch craft-com.ubuntu.cloud-buildd:core22 local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 --config 'raw.idmap=both 1000 0' --config security.syscalls.intercept.mknod=true --config user.craft_providers.status=STARTING --config user.craft_providers.timer=2024-05-07T11:37:51.498040+00:00 --config user.craft_providers.pid=4094036
2024-05-07 13:37:55.150 Successfully launched instance base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861.
2024-05-07 13:37:55.150 Executing on host: lxc --project snapcraft config get local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.status
2024-05-07 13:37:55.234 Setting up base instance 'base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861'
2024-05-07 13:37:55.234 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.status PREPARING
2024-05-07 13:37:55.305 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:37:55.305270+00:00
2024-05-07 13:37:55.306 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-05-07 13:37:55.396 Set instance timer to '2024-05-07T11:37:55.305270+00:00'
2024-05-07 13:37:55.422 Executing on host: lxc --project snapcraft file push /var/tmp/tmpdl9lohlz local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/craft-instance.conf --mode=0644
2024-05-07 13:37:55.532 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/craft-instance.conf
2024-05-07 13:37:55.707 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical cat /etc/os-release
2024-05-07 13:37:55.889 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-05-07 13:37:56.054 Executing on host: lxc --project snapcraft file pull local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/craft-instance.conf /home/jw/tmpdud_0h_g.tmp-craft/tmprsuj5cqs
2024-05-07 13:37:56.151 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-05-07 13:37:56.266 Executing on host: lxc --project snapcraft file pull local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/craft-instance.conf /home/jw/tmpqu95h4g5.tmp-craft/tmpgekuw8el
2024-05-07 13:37:56.320 Executing on host: lxc --project snapcraft file push /var/tmp/tmptatzzsy0 local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/craft-instance.conf --mode=0644
2024-05-07 13:37:56.374 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/craft-instance.conf
2024-05-07 13:37:56.440 Executing on host: lxc --project snapcraft file push /var/tmp/tmpek6lr5_y local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/environment --mode=0644
2024-05-07 13:37:56.493 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/environment
2024-05-07 13:37:56.559 Executing on host: lxc --project snapcraft file push /var/tmp/tmpyuwzyg_b local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/apt/apt.conf.d/20auto-upgrades --mode=0644
2024-05-07 13:37:56.611 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/apt/apt.conf.d/20auto-upgrades
2024-05-07 13:37:56.676 Waiting for environment to be ready...
2024-05-07 13:37:56.676 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-05-07 13:37:56.745 Executing on host: lxc --project snapcraft file push /var/tmp/tmplha6kzdh local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/hostname --mode=0644
2024-05-07 13:37:56.796 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/hostname
2024-05-07 13:37:56.933 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical hostname -F /etc/hostname
2024-05-07 13:37:57.093 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
2024-05-07 13:37:57.236 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl enable systemd-resolved
2024-05-07 13:37:57.376 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl restart systemd-resolved
2024-05-07 13:37:57.608 Executing on host: lxc --project snapcraft file push /var/tmp/tmppj0yc2j_ local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/systemd/network/10-eth0.network --mode=0644
2024-05-07 13:37:57.662 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/systemd/network/10-eth0.network
2024-05-07 13:37:57.800 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl enable systemd-networkd
2024-05-07 13:37:58.026 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl restart systemd-networkd
2024-05-07 13:37:58.247 Waiting for networking to be ready...
2024-05-07 13:37:58.247 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical getent hosts snapcraft.io
2024-05-07 13:37:58.397 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:37:58.397248+00:00
2024-05-07 13:37:58.420 Executing on host: lxc --project snapcraft file push /var/tmp/tmpa_q1ogi4 local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/apt/apt.conf.d/00no-recommends --mode=0644
2024-05-07 13:37:58.497 Set instance timer to '2024-05-07T11:37:58.397248+00:00'
2024-05-07 13:37:58.580 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/apt/apt.conf.d/00no-recommends
2024-05-07 13:37:58.737 Executing on host: lxc --project snapcraft file push /var/tmp/tmpm3airou5 local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861/etc/apt/apt.conf.d/00update-errors --mode=0644
2024-05-07 13:37:58.828 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /etc/apt/apt.conf.d/00update-errors
2024-05-07 13:37:58.965 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical apt-get update
2024-05-07 13:38:01.501 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:01.501071+00:00
2024-05-07 13:38:01.597 Set instance timer to '2024-05-07T11:38:01.501071+00:00'
2024-05-07 13:38:04.600 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:04.600423+00:00
2024-05-07 13:38:04.700 Set instance timer to '2024-05-07T11:38:04.600423+00:00'
2024-05-07 13:38:07.704 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:07.704186+00:00
2024-05-07 13:38:07.799 Set instance timer to '2024-05-07T11:38:07.704186+00:00'
2024-05-07 13:38:10.801 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:10.801191+00:00
2024-05-07 13:38:10.894 Set instance timer to '2024-05-07T11:38:10.801191+00:00'
2024-05-07 13:38:13.897 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:13.897520+00:00
2024-05-07 13:38:14.010 Set instance timer to '2024-05-07T11:38:13.897520+00:00'
2024-05-07 13:38:17.013 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:17.013454+00:00
2024-05-07 13:38:17.130 Set instance timer to '2024-05-07T11:38:17.013454+00:00'
2024-05-07 13:38:20.133 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:20.133195+00:00
2024-05-07 13:38:20.223 Set instance timer to '2024-05-07T11:38:20.133195+00:00'
2024-05-07 13:38:23.225 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:23.225483+00:00
2024-05-07 13:38:23.321 Set instance timer to '2024-05-07T11:38:23.225483+00:00'
2024-05-07 13:38:26.325 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:26.325196+00:00
2024-05-07 13:38:26.441 Set instance timer to '2024-05-07T11:38:26.325196+00:00'
2024-05-07 13:38:29.444 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:29.444505+00:00
2024-05-07 13:38:29.549 Set instance timer to '2024-05-07T11:38:29.444505+00:00'
2024-05-07 13:38:32.553 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:32.553057+00:00
2024-05-07 13:38:32.661 Set instance timer to '2024-05-07T11:38:32.553057+00:00'
2024-05-07 13:38:35.665 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:35.665065+00:00
2024-05-07 13:38:35.754 Set instance timer to '2024-05-07T11:38:35.665065+00:00'
2024-05-07 13:38:38.757 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:38.757401+00:00
2024-05-07 13:38:38.769 Executing in container: lxc --project snapcraft exec local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical bash -c 'exec 3<> /dev/tcp/snapcraft.io/443'
2024-05-07 13:38:38.874 Set instance timer to '2024-05-07T11:38:38.757401+00:00'
2024-05-07 13:38:41.877 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:41.877501+00:00
2024-05-07 13:38:41.962 Set instance timer to '2024-05-07T11:38:41.877501+00:00'
2024-05-07 13:38:44.966 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:44.965896+00:00
2024-05-07 13:38:45.077 Set instance timer to '2024-05-07T11:38:44.965896+00:00'
2024-05-07 13:38:48.080 Executing on host: lxc --project snapcraft config set local:base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 user.craft_providers.timer 2024-05-07T11:38:48.080360+00:00
2024-05-07 13:38:48.184 Set instance timer to '2024-05-07T11:38:48.080360+00:00'
2024-05-07 13:38:48.780 craft-providers error: A network related operation failed in a context of no network access.
Verify that the environment has internet connectivity; see https://craft-providers.readthedocs.io/ for further reference.
2024-05-07 13:38:48.780 Full execution log: '/home/jw/.local/state/snapcraft/log/snapcraft-20240507-133748.717420.log'

mr_cal · May 7, 2024, 2:19pm

Looks like apt-get update is failing inside the container.

Unfortunately, snapcraft does not capture the output of that command when it fails (bug here).

One option is to launch the container and try updating to see the error:

lxc --project snapcraft start base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861
lxc --project snapcraft exec base-instance-snapcraft-buildd-base-v71--d799df4d4588b4abe861 bash
apt-get update

joew · May 7, 2024, 9:25pm

You’re righ, @mr_cal: apt-get update is hanging.

What might be the reason?

Unfortunately, all the commands needed to investigate network problems seem to be missing. No ping, no ip, no dig…

Could find only networkctl, but:

root@snapcraft-subiquity-on-amd64-for-amd64-93859022:/# networkctl list
Failed to connect system bus: No such file or directory
root@snapcraft-subiquity-on-amd64-for-amd64-93859022:/#

So how to find the reason for the problem?

mr_cal · May 7, 2024, 9:55pm

One available command is getent. Snapcraft should call getent hosts snapcraft.io to verify network connectivity before calling apt-get update, so it could be more specific than a broad network connectivity problem.

You can narrow down the problem slightly more by launching a new lxd container:

lxc launch ubuntu:24.04 test-env
lxc exec test-env bash
> getent hosts snapcraft.io
> apt-get update

It could be because you have docker installed, a firewall, VPN, or proxy configured.

If it’s none of those things, can you paste the output of running apt-get update from inside the LXD container?

joew · May 7, 2024, 10:45pm

Result is the same as in the subiquity instance. Thus the problem is not specific to subiquity.

root@test-env:~# getent hosts snapcraft.io
2620:2d:4000:1::27 snapcraft.io
2620:2d:4000:1::26 snapcraft.io
2620:2d:4000:1::28 snapcraft.io
root@test-env:~# apt-get update
Ign:1 http://security.ubuntu.com/ubuntu noble-security InRelease
Ign:2 http://archive.ubuntu.com/ubuntu noble InRelease
Ign:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Ign:4 http://archive.ubuntu.com/ubuntu noble-backports InRelease
Ign:1 http://security.ubuntu.com/ubuntu noble-security InRelease
Ign:2 http://archive.ubuntu.com/ubuntu noble InRelease
Ign:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Ign:4 http://archive.ubuntu.com/ubuntu noble-backports InRelease
Ign:1 http://security.ubuntu.com/ubuntu noble-security InRelease
Ign:2 http://archive.ubuntu.com/ubuntu noble InRelease
Ign:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Ign:4 http://archive.ubuntu.com/ubuntu noble-backports InRelease
Err:1 http://security.ubuntu.com/ubuntu noble-security InRelease
  Could not connect to security.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (2620:2d:4002:1::103). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (2620:2d:4002:1::101). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (91.189.91.83), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.82), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.81), connection timed out
Err:2 http://archive.ubuntu.com/ubuntu noble InRelease
  Could not connect to archive.ubuntu.com:80 (2620:2d:4002:1::103). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (2620:2d:4002:1::101). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.81), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.83), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.82), connection timed out
Err:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease
  Unable to connect to archive.ubuntu.com:http:
Err:4 http://archive.ubuntu.com/ubuntu noble-backports InRelease
  Unable to connect to archive.ubuntu.com:http:
Reading package lists... Done
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/noble/InRelease  Could not connect to archive.ubuntu.com:80 (2620:2d:4002:1::103). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (2620:2d:4002:1::101). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Could not connect to archive.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.81), connection timed out Could not connect to archive.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.83), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.91.82), connection timed out
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/noble-updates/InRelease  Unable to connect to archive.ubuntu.com:http:
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/noble-backports/InRelease  Unable to connect to archive.ubuntu.com:http:
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/noble-security/InRelease  Could not connect to security.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (2620:2d:4002:1::103). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (2620:2d:4002:1::101). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (91.189.91.83), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.36), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.82), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.81), connection timed out
W: Some index files failed to download. They have been ignored, or old ones used instead.
root@test-env:~# tracepath web.de
 1?: [LOCALHOST]                      pmtu 1500
 1:  _gateway.lxd                                          0.081ms
 1:  _gateway.lxd                                          0.041ms
 2:  no reply
 3:  no reply
 4:  no reply
 5:  no reply
 6:  no reply
 7:  no reply
 8:  no reply
 9:  no reply
10:  no reply
11:  no reply
12:  no reply
13:  no reply
14:  no reply
^C
root@test-env:~#

Docker is uninstalled (and rebooted after uninstalling)
host-firewall is disabled (firewall on router (pfsense) should not be a problem, I guess)
No VPN active
No proxy configured

mr_cal · May 8, 2024, 3:06pm

Thanks for the logs. You have connectivity inside the container and DNS is working, but connections to the archives are failing. I’m not sure what the problem is now.

Does anyone else here have any ideas?

It may be worthwhile to try reaching the archives (ping, nc -vz archive.ubuntu.com 80, nc -vz security.ubuntu.com 80, etc)

PS - as you probably noticed when calling tracepath, the ubuntu:24.04 image has a few more pre-installed utilities than the stripped-down container that snapcraft launches.