Snapcraft pull stage seems to have wrong directory (snapcraft 8.3.1)

Charlee · August 2, 2024, 10:30am

Hello,

I am trying to build a snap on an azure VM and somehow behavior seems different than my local machine.

I am testing with the simplest snap possible; it packages some scripts with help of the dump plugin:

name: conf-hardware
base: core22
version: 'dirty'
summary: Snap that configures hardware
description: |
    Snap for configuring hardware like audio,display settings,etc

grade: stable
confinement: strict

apps:
  audio-configuration:
    command: configure_audio.sh
    daemon: simple

parts:
  basic:
    source: scripts
    plugin: dump
    source-type: local

My directory structure is like this:

.
├── README.md
├── scripts
│   ├── configure_audio.sh
└── snap
    ├── hooks
    │   └── connect-plug-audio-config
    └── snapcraft.yaml

When I build this locally (just call snapcraft from the directory) everything seems happy.

When I try to build the exact same code on my VM however, I get the following error: Failed to pull source: 'scripts' not found.

Inspecting the pull step by running snapcraft pull basic --shell I see that the pull step is executed in the very root of the project (“/”), rather than in the project dir, like is the case when I do the build on my own device.

That is, on the VM snapcraft pull basic --shell gives me this output:

$ snapcraft pull basic --shell
Launching shell on build environment...                                                            
snapcraft-conf-hardware-on-amd64-for-amd64-516127 /# pwd
/
snapcraft-conf-hardware-on-amd64-for-amd64-516127 /# ls
bin   build  etc   lib	  lib64   media  opt   root  sbin  srv	tmp  var
boot  dev    home  lib32  libx32  mnt	 proc  run   snap  sys	usr

Whereas on ny own device it looks like this:

$ snapcraft pull basic --shell
Launching shell on build environment...                                                             
snapcraft-conf-hardware-on-amd64-for-amd64-1199677 ../project# pwd
/root/project
snapcraft-conf-hardware-on-amd64-for-amd64-1199677 ../project# ls
README.md  scripts  snap

Is there some setting automatially applied that can cause this?

I did notice when inspecting the snapcraft logs that on my local device there is a line saying that it is falling back form craft-application to `snapcraft:

2024-08-02 12:27:06.824 :: 2024-08-02 12:26:24.362 Starting snapcraft, version 8.3.1
2024-08-02 12:27:06.824 :: 2024-08-02 12:26:24.362 Log verbosity level set to BRIEF
2024-08-02 12:27:06.824 :: 2024-08-02 12:26:24.362 Falling back from craft-application to snapcraft
2024-08-02 12:27:06.824 :: 2024-08-02 12:26:24.363 lifecycle command: 'pull', arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=False, manifest_image_information=None, bind_ssh=False, build_for='amd64', http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, parts=['basic'], shell=True, shell_after=False)
2024-08-02 12:27:06.824 :: 2024-08-02 12:26:24.363 command: pull, arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=False, manifest_image_information=None, bind_ssh=False, build_for='amd64', http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, parts=['basic'], shell=True, shell_after=False)
2024-08-02 12:27:06.825 :: 2024-08-02 12:26:24.367 CPU count (from process affinity): 12
2024-08-02 12:27:06.825 :: 2024-08-02 12:26:24.367 Invalid SNAPCRAFT_MAX_PARALLEL_BUILD_COUNT ''
2024-08-02 12:27:06.825 :: 2024-08-02 12:26:24.367 Running on amd64 for amd64
2024-08-02 12:27:06.825 :: 2024-08-02 12:26:24.371 Initializing parts lifecycle
...

Whereas the VM logs do not say something similar:

2024-08-02 10:24:36.389 :: 2024-08-02 10:18:57.663 Starting snapcraft, version 8.3.1
2024-08-02 10:24:36.389 :: 2024-08-02 10:18:57.664 Log verbosity level set to BRIEF
2024-08-02 10:24:36.389 :: 2024-08-02 10:18:57.664 Configuring application...
2024-08-02 10:24:36.389 :: 2024-08-02 10:18:57.664 Preparing application...
2024-08-02 10:24:36.389 :: 2024-08-02 10:18:57.665 Build plan: platform=None, build_for=amd64
2024-08-02 10:24:36.389 :: 2024-08-02 10:18:57.666 Loading project file '/root/project/snap/snapcraft.yaml'
2024-08-02 10:24:36.389 :: 2024-08-02 10:18:57.675 Processing grammar (on amd64 for amd64)
2024-08-02 10:24:36.389 :: 2024-08-02 10:18:57.675 Processing grammar for source: scripts
2024-08-02 10:24:36.390 :: 2024-08-02 10:18:57.675 Processing grammar for plugin: dump
2024-08-02 10:24:36.390 :: 2024-08-02 10:18:57.675 Processing grammar for source-type: local
2024-08-02 10:24:36.390 :: 2024-08-02 10:18:57.678 Initialising lifecycle manager in /root

Any ideas?

ogra · August 2, 2024, 11:44am

Have you tried without setting source-type ?

Charlee · August 2, 2024, 11:45am

Yes, then it says it cannot infer the source type (because it cannot find the path)

ogra · August 2, 2024, 11:50am

Hmm, in both cases (VM as well as local)?

Charlee · August 2, 2024, 11:56am

No only on VM. The issue is the same reason I think, the fact that somehow the working directory is “/” and not /root/project, where the scripts folder is located

ogra · August 2, 2024, 11:57am

Yeah, that clearly points to a snapcraft bug…

Charlee · August 2, 2024, 12:02pm

Just a mystery how this bug is only triggered on the VM

Any ideas on what environmental differences here could be triggering different behavior?

Do these logs ring any bells? Why does my local version do a snapcraft fallback while the remote one does not? Does that mean it is building a craft-application? Is that something different? Are the path rules different? (the revision of snapcraft is 12136 for both)

mr_cal · August 2, 2024, 1:29pm

The logs from your desktop indicate it’s trying to build a core22 snap whereas the logs in the Azure VM indicate it’s trying to build a core24 snap. I haven’t seen this behavior before.

Failed to pull source: 'scripts' not found.

This error makes me suspect the project directory is not being properly mounted in the LXD container. Perhaps a file system, permission, or ownership issue.

Can you post the full logs for the two scenarios? That would be very helpful.

Charlee · August 2, 2024, 1:47pm

When I do the --shell option I can cd into /root/project, so the path does exist… Not sure if that helps with the hypothesis about mounting issues.

I can put full logs later

aliihsan · August 9, 2024, 8:18am

Permission of directories

gitlab-runner@clp-dev-gitlab-runner-snap-build-weu:~/builds/zp7KXvBs/0/claysolutions/clp/embedded/snaps/salto-conf-hardware$ ls -al
total 32
drwxr-x--- 5 gitlab-runner users 4096 Aug  9 08:11 .
drwxr-x--- 6 gitlab-runner users 4096 Aug  9 08:11 ..
drwxr-x--- 5 gitlab-runner users 4096 Aug  9 08:11 .git
-rw-r----- 1 gitlab-runner users   12 Aug  9 08:11 .gitignore
-rw-r----- 1 gitlab-runner users  390 Aug  9 08:11 .gitlab-ci.yml
-rw-r----- 1 gitlab-runner users 2232 Aug  9 08:11 README.md
drwxr-x--- 2 gitlab-runner users 4096 Aug  9 08:11 scripts
drwxr-x--- 3 gitlab-runner users 4096 Aug  9 08:11 snap

Debug logs

gitlab-runner@clp-dev-gitlab-runner-snap-build-weu:~/builds/zp7KXvBs/0/claysolutions/clp/embedded/snaps/salto-conf-hardware$ snapcraft --verbosity=debug
2024-08-09 08:04:41.089 Starting snapcraft, version 8.3.2
2024-08-09 08:04:41.089 Logging execution to '/home/gitlab-runner/.local/state/snapcraft/log/snapcraft-20240809-080441.088237.log'
2024-08-09 08:04:41.091 lifecycle command: 'pack', arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=False, manifest_image_information=None, bind_ssh=False, build_for=None, http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, directory=None, output=None)
2024-08-09 08:04:41.091 command: pack, arguments: Namespace(destructive_mode=False, use_lxd=False, debug=False, enable_manifest=False, manifest_image_information=None, bind_ssh=False, build_for=None, http_proxy=None, https_proxy=None, ua_token=None, enable_experimental_ua_services=False, enable_experimental_plugins=False, enable_experimental_extensions=False, enable_developer_debug=False, enable_experimental_target_arch=False, target_arch=None, provider=None, directory=None, output=None)
2024-08-09 08:04:41.100 CPU count (from process affinity): 2
2024-08-09 08:04:41.100 Invalid SNAPCRAFT_MAX_PARALLEL_BUILD_COUNT ''
2024-08-09 08:04:41.101 Running on amd64 for amd64
2024-08-09 08:04:41.103 Checking build provider availability
2024-08-09 08:04:41.112 Retrieved snap config: {}
2024-08-09 08:04:41.112 Using default provider 'lxd' on linux system.
2024-08-09 08:04:41.202 Executing on host: lxc --project default profile show local:default
2024-08-09 08:04:41.252 Using hostname 'snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736'
2024-08-09 08:04:41.252 Launching instance...
2024-08-09 08:04:41.253 Executing on host: lxc remote list --format=yaml
2024-08-09 08:04:41.296 Remote 'craft-com.ubuntu.cloud-buildd' already exists.
2024-08-09 08:04:41.296 Executing on host: lxc project list local: --format=yaml
2024-08-09 08:04:41.346 Set LXD instance name to 'snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736'
2024-08-09 08:04:41.347 Checking for instance 'snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736' in project 'snapcraft' in remote 'local'
2024-08-09 08:04:41.347 Executing on host: lxc --project snapcraft list local: --format=yaml
2024-08-09 08:04:41.558 Executing on host: lxc --project snapcraft config get local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 raw.idmap
2024-08-09 08:04:41.604 Executing on host: lxc --project snapcraft list local: --format=yaml
2024-08-09 08:04:41.762 Instance exists and is not running. Starting instance.
2024-08-09 08:04:41.762 Starting instance
2024-08-09 08:04:41.762 Executing on host: lxc --project snapcraft start local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736
2024-08-09 08:04:42.090 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-08-09 08:04:42.260 Executing on host: lxc --project snapcraft file pull local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736/etc/craft-instance.conf /home/gitlab-runner/tmp50u0n34p.tmp-craft/tmp821lbr84
2024-08-09 08:04:42.394 Instance has already been setup.
2024-08-09 08:04:42.395 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical cat /etc/os-release
2024-08-09 08:04:42.518 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-08-09 08:04:42.639 Executing on host: lxc --project snapcraft file pull local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736/etc/craft-instance.conf /home/gitlab-runner/tmp3iysr5qg.tmp-craft/tmp3hh0frn8
2024-08-09 08:04:42.707 Instance is compatible with compatibility tag 'snapcraft-buildd-base-v7.1'
2024-08-09 08:04:42.708 No cache path set, not mounting cache directories.
2024-08-09 08:04:42.708 Waiting for environment to be ready...
2024-08-09 08:04:42.709 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-08-09 08:04:42.839 systemctl is-system-running status: starting
2024-08-09 08:04:43.089 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-08-09 08:04:43.237 systemctl is-system-running status: starting
2024-08-09 08:04:43.488 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-08-09 08:04:43.615 systemctl is-system-running status: starting
2024-08-09 08:04:43.865 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-08-09 08:04:43.951 systemctl is-system-running status: starting
2024-08-09 08:04:44.202 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-08-09 08:04:44.296 systemctl is-system-running status: starting
2024-08-09 08:04:44.547 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-08-09 08:04:44.639 systemctl is-system-running status: starting
2024-08-09 08:04:44.889 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical systemctl is-system-running
2024-08-09 08:04:44.978 Waiting for networking to be ready...
2024-08-09 08:04:44.979 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical getent hosts snapcraft.io
2024-08-09 08:04:45.076 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical snap unset system proxy.http
2024-08-09 08:04:45.297 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical snap unset system proxy.https
2024-08-09 08:04:45.515 Installing snap 'snapcraft' with channel=None and classic=True
2024-08-09 08:04:45.515 Installing snap 'snapcraft' from host (classic=True)
2024-08-09 08:04:45.521 Installing base snap 'core22' for 'snapcraft' from host
2024-08-09 08:04:45.521 Installing snap 'core22' from host (classic=False)
2024-08-09 08:04:45.523 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-08-09 08:04:45.639 Executing on host: lxc --project snapcraft file pull local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736/etc/craft-instance.conf /home/gitlab-runner/tmpvcul1z96.tmp-craft/tmpi1q4kkak
2024-08-09 08:04:45.686 Revisions found: host='1439', target='1439'
2024-08-09 08:04:45.686 Skipping snap injection: target is already up-to-date with revision on host
2024-08-09 08:04:45.687 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /etc/craft-instance.conf
2024-08-09 08:04:45.802 Executing on host: lxc --project snapcraft file pull local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736/etc/craft-instance.conf /home/gitlab-runner/tmpwwkvbey9.tmp-craft/tmpoeyjqbp_
2024-08-09 08:04:45.850 Revisions found: host='12378', target='12378'
2024-08-09 08:04:45.851 Skipping snap injection: target is already up-to-date with revision on host
2024-08-09 08:04:45.851 Executing on host: lxc --project snapcraft config device show local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736
2024-08-09 08:04:45.895 Executing on host: lxc --project snapcraft config device add local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 disk-/root/project disk source=/home/gitlab-runner/builds/zp7KXvBs/0/claysolutions/clp/embedded/snaps/salto-conf-hardware path=/root/project
2024-08-09 08:04:45.980 Executing on host: lxc --project snapcraft file push /tmp/tmpjwc4pay1 local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736/root/.bashrc --mode=644
2024-08-09 08:04:46.025 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical chown root:root /root/.bashrc
2024-08-09 08:04:46.116 Emitter: Pausing control of the terminal
2024-08-09 08:04:50.041 Starting snapcraft, version 8.3.2
2024-08-09 08:04:50.041 Logging execution to '/tmp/snapcraft.log'
2024-08-09 08:04:50.041 Configuring application...
2024-08-09 08:04:50.042 Preparing application...
2024-08-09 08:04:50.043 Build plan: platform=None, build_for=amd64
2024-08-09 08:04:50.043 Loading project file '/root/project/snap/snapcraft.yaml'
2024-08-09 08:04:50.055 Processing grammar (on amd64 for amd64)
2024-08-09 08:04:50.055 Processing grammar for source: scripts
2024-08-09 08:04:50.055 Processing grammar for source-type: local
2024-08-09 08:04:50.055 Processing grammar for plugin: dump
2024-08-09 08:04:50.058 Initialising lifecycle manager in /root
2024-08-09 08:04:50.059 Project vars: {'version': 'dirty', 'grade': 'stable'}
2024-08-09 08:04:50.059 Adopting part: None
2024-08-09 08:04:50.060 Using parallel build count of 2 from CPU count
2024-08-09 08:04:50.060 is_snap: True, SNAP_NAME set to snapcraft
2024-08-09 08:04:50.061 process basic:Step.PULL
2024-08-09 08:04:50.061 add action basic:Step.PULL(ActionType.RUN)
2024-08-09 08:04:50.062 process basic:Step.BUILD
2024-08-09 08:04:50.062 add action basic:Step.BUILD(ActionType.RUN)
2024-08-09 08:04:50.062 process basic:Step.STAGE
2024-08-09 08:04:50.062 add action basic:Step.STAGE(ActionType.RUN)
2024-08-09 08:04:50.063 process basic:Step.PRIME
2024-08-09 08:04:50.063 add action basic:Step.PRIME(ActionType.RUN)
2024-08-09 08:04:50.063 Initialising lifecycle
2024-08-09 08:04:50.064 ignore patterns: ['*.snap']
2024-08-09 08:04:50.064 Installing build-packages
2024-08-09 08:04:50.064 Installing build-snaps
2024-08-09 08:04:50.280 verify plugin environment for part 'basic'
2024-08-09 08:04:50.282 Pulling basic
2024-08-09 08:04:50.282 execute action basic:Action(part_name='basic', step=Step.PULL, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
2024-08-09 08:04:50.283 remove directory /root/parts/basic/src
2024-08-09 08:04:50.284 Failed to pull source: 'scripts' not found.
2024-08-09 08:04:50.288 Traceback (most recent call last):
2024-08-09 08:04:50.289   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_application/services/lifecycle.py", line 261, in run
2024-08-09 08:04:50.289     aex.execute(action, stdout=stream, stderr=stream)
2024-08-09 08:04:50.289   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 327, in execute
2024-08-09 08:04:50.289     self._executor.execute(actions, stdout=stdout, stderr=stderr)
2024-08-09 08:04:50.289   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 137, in execute
2024-08-09 08:04:50.290     self._run_action(act, stdout=stdout, stderr=stderr)
2024-08-09 08:04:50.290   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 214, in _run_action
2024-08-09 08:04:50.290     handler.run_action(action, stdout=stdout, stderr=stderr)
2024-08-09 08:04:50.290   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 170, in run_action
2024-08-09 08:04:50.290     state = handler(step_info, stdout=stdout, stderr=stderr)
2024-08-09 08:04:50.290   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 196, in _run_pull
2024-08-09 08:04:50.290     self._run_step(
2024-08-09 08:04:50.291   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 491, in _run_step
2024-08-09 08:04:50.291     return step_handler.run_builtin()
2024-08-09 08:04:50.291   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/step_handler.py", line 108, in run_builtin
2024-08-09 08:04:50.291     return handler()
2024-08-09 08:04:50.291   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/step_handler.py", line 112, in _builtin_pull
2024-08-09 08:04:50.291     self._source_handler.pull()
2024-08-09 08:04:50.292   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/sources/local_source.py", line 80, in pull
2024-08-09 08:04:50.292     raise errors.SourceNotFound(self.source)
2024-08-09 08:04:50.292 craft_parts.sources.errors.SourceNotFound: Failed to pull source: 'scripts' not found.
2024-08-09 08:04:50.292 Make sure the source path is correct and accessible.
2024-08-09 08:04:50.292 Recommended resolution: Make sure the source path is correct and accessible.
2024-08-09 08:04:50.552 Emitter: Resuming control of the terminal
2024-08-09 08:04:50.553 Executing in container: lxc --project snapcraft exec local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CRAFT_MANAGED_MODE=1 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical test -f /tmp/snapcraft.log
2024-08-09 08:04:50.668 Executing on host: lxc --project snapcraft file pull local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736/tmp/snapcraft.log /home/gitlab-runner/tmpcyzcefda.tmp-craft/tmpobc3zdnp
2024-08-09 08:04:50.715 Logs retrieved from managed instance:
2024-08-09 08:04:50.716 :: 2024-08-09 08:04:50.041 Starting snapcraft, version 8.3.2
2024-08-09 08:04:50.716 :: 2024-08-09 08:04:50.041 Log verbosity level set to BRIEF
2024-08-09 08:04:50.716 :: 2024-08-09 08:04:50.041 Configuring application...
2024-08-09 08:04:50.716 :: 2024-08-09 08:04:50.042 Preparing application...
2024-08-09 08:04:50.716 :: 2024-08-09 08:04:50.043 Build plan: platform=None, build_for=amd64
2024-08-09 08:04:50.716 :: 2024-08-09 08:04:50.043 Loading project file '/root/project/snap/snapcraft.yaml'
2024-08-09 08:04:50.717 :: 2024-08-09 08:04:50.055 Processing grammar (on amd64 for amd64)
2024-08-09 08:04:50.717 :: 2024-08-09 08:04:50.055 Processing grammar for source: scripts
2024-08-09 08:04:50.717 :: 2024-08-09 08:04:50.055 Processing grammar for source-type: local
2024-08-09 08:04:50.717 :: 2024-08-09 08:04:50.055 Processing grammar for plugin: dump
2024-08-09 08:04:50.717 :: 2024-08-09 08:04:50.058 Initialising lifecycle manager in /root
2024-08-09 08:04:50.717 :: 2024-08-09 08:04:50.059 Project vars: {'version': 'dirty', 'grade': 'stable'}
2024-08-09 08:04:50.717 :: 2024-08-09 08:04:50.059 Adopting part: None
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.060 Using parallel build count of 2 from CPU count
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.060 is_snap: True, SNAP_NAME set to snapcraft
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.061 process basic:Step.PULL
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.061 add action basic:Step.PULL(ActionType.RUN)
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.062 process basic:Step.BUILD
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.062 add action basic:Step.BUILD(ActionType.RUN)
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.062 process basic:Step.STAGE
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.062 add action basic:Step.STAGE(ActionType.RUN)
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.063 process basic:Step.PRIME
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.063 add action basic:Step.PRIME(ActionType.RUN)
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.063 Initialising lifecycle
2024-08-09 08:04:50.718 :: 2024-08-09 08:04:50.064 ignore patterns: ['*.snap']
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.064 Installing build-packages
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.064 Installing build-snaps
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.280 verify plugin environment for part 'basic'
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.282 Pulling basic
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.282 execute action basic:Action(part_name='basic', step=Step.PULL, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.283 remove directory /root/parts/basic/src
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.284 Failed to pull source: 'scripts' not found.
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.288 Traceback (most recent call last):
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.289   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_application/services/lifecycle.py", line 261, in run
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.289     aex.execute(action, stdout=stream, stderr=stream)
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.289   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 327, in execute
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.289     self._executor.execute(actions, stdout=stdout, stderr=stderr)
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.289   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 137, in execute
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.290     self._run_action(act, stdout=stdout, stderr=stderr)
2024-08-09 08:04:50.719 :: 2024-08-09 08:04:50.290   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/executor.py", line 214, in _run_action
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.290     handler.run_action(action, stdout=stdout, stderr=stderr)
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.290   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 170, in run_action
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.290     state = handler(step_info, stdout=stdout, stderr=stderr)
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.290   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 196, in _run_pull
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.290     self._run_step(
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.291   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/part_handler.py", line 491, in _run_step
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.291     return step_handler.run_builtin()
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.291   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/step_handler.py", line 108, in run_builtin
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.291     return handler()
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.291   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/executor/step_handler.py", line 112, in _builtin_pull
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.291     self._source_handler.pull()
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.292   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_parts/sources/local_source.py", line 80, in pull
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.292     raise errors.SourceNotFound(self.source)
2024-08-09 08:04:50.720 :: 2024-08-09 08:04:50.292 craft_parts.sources.errors.SourceNotFound: Failed to pull source: 'scripts' not found.
2024-08-09 08:04:50.721 :: 2024-08-09 08:04:50.292 Make sure the source path is correct and accessible.
2024-08-09 08:04:50.721 :: 2024-08-09 08:04:50.292 Recommended resolution: Make sure the source path is correct and accessible.
2024-08-09 08:04:50.721 Executing on host: lxc --project snapcraft config device show local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736
2024-08-09 08:04:50.765 Executing on host: lxc --project snapcraft config device remove local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736 disk-/root/project
2024-08-09 08:04:50.858 Executing on host: lxc --project snapcraft stop local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736
2024-08-09 08:04:55.016 Failed to execute pack in instance.
2024-08-09 08:04:55.018 Traceback (most recent call last):
2024-08-09 08:04:55.018   File "/snap/snapcraft/12378/lib/python3.10/site-packages/snapcraft/application.py", line 351, in main
2024-08-09 08:04:55.019     return app.run()
2024-08-09 08:04:55.019   File "/snap/snapcraft/12378/lib/python3.10/site-packages/snapcraft/application.py", line 179, in run
2024-08-09 08:04:55.019     return_code = super().run()
2024-08-09 08:04:55.019   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_application/application.py", line 492, in run
2024-08-09 08:04:55.019     dispatcher = self._get_dispatcher()
2024-08-09 08:04:55.019   File "/snap/snapcraft/12378/lib/python3.10/site-packages/snapcraft/application.py", line 298, in _get_dispatcher
2024-08-09 08:04:55.019     raise errors.ClassicFallback()
2024-08-09 08:04:55.019 snapcraft.errors.ClassicFallback
2024-08-09 08:04:55.019
2024-08-09 08:04:55.019 During handling of the above exception, another exception occurred:
2024-08-09 08:04:55.019 Traceback (most recent call last):
2024-08-09 08:04:55.019   File "/snap/snapcraft/12378/lib/python3.10/site-packages/snapcraft/parts/lifecycle.py", line 542, in _run_in_provider
2024-08-09 08:04:55.019     instance.execute_run(cmd, check=True, cwd=output_dir)
2024-08-09 08:04:55.020   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_providers/lxd/lxd_instance.py", line 293, in execute_run
2024-08-09 08:04:55.020     return self.lxc.exec(
2024-08-09 08:04:55.020   File "/snap/snapcraft/12378/lib/python3.10/site-packages/craft_providers/lxd/lxc.py", line 387, in exec
2024-08-09 08:04:55.020     return runner(final_cmd, timeout=timeout, check=check, **kwargs)
2024-08-09 08:04:55.020   File "/snap/snapcraft/12378/usr/lib/python3.10/subprocess.py", line 526, in run
2024-08-09 08:04:55.020     raise CalledProcessError(retcode, process.args,
2024-08-09 08:04:55.020 subprocess.CalledProcessError: Command '['lxc', '--project', 'snapcraft', 'exec', 'local:snapcraft-salto-conf-hardware-on-amd64-for-amd64-531736', '--cwd', '/root/project', '--', 'env', 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin', 'CRAFT_MANAGED_MODE=1', 'DEBIAN_FRONTEND=noninteractive', 'DEBCONF_NONINTERACTIVE_SEEN=true', 'DEBIAN_PRIORITY=critical', 'snapcraft', 'pack', '--verbosity=debug', '--build-for', 'amd64']' returned non-zero exit status 1.
2024-08-09 08:04:55.020 Recommended resolution: Run the same command again with --debug to shell into the environment if you wish to introspect this failure.

Workarround was giving full permission to CWD .

gitlab-runner@clp-dev-gitlab-runner-snap-build-weu:~/builds/zp7KXvBs/0/claysolutions/clp/embedded/snaps/salto-conf-hardware$ snapcraft clean
Cleaned build provider
gitlab-runner@clp-dev-gitlab-runner-snap-build-weu:~/builds/zp7KXvBs/0/claysolutions/clp/embedded/snaps/salto-conf-hardware$ chmod 777 .
gitlab-runner@clp-dev-gitlab-runner-snap-build-weu:~/builds/zp7KXvBs/0/claysolutions/clp/embedded/snaps/salto-conf-hardware$ ls -al
total 32
drwxrwxrwx 5 gitlab-runner users 4096 Aug  9 08:11 .
drwxr-x--- 6 gitlab-runner users 4096 Aug  9 08:11 ..
drwxr-x--- 5 gitlab-runner users 4096 Aug  9 08:11 .git
-rw-r----- 1 gitlab-runner users   12 Aug  9 08:11 .gitignore
-rw-r----- 1 gitlab-runner users  390 Aug  9 08:11 .gitlab-ci.yml
-rw-r----- 1 gitlab-runner users 2232 Aug  9 08:11 README.md
drwxr-x--- 2 gitlab-runner users 4096 Aug  9 08:11 scripts
drwxr-x--- 3 gitlab-runner users 4096 Aug  9 08:11 snap
gitlab-runner@clp-dev-gitlab-runner-snap-build-weu:~/builds/zp7KXvBs/0/claysolutions/clp/embedded/snaps/salto-conf-hardware$ snapcraft
Generated snap metadata
Created snap package salto-conf-hardware_dirty_amd64.snap
gitlab-runner@clp-dev-gitlab-runner-snap-build-weu:~/builds/zp7KXvBs/0/claysolutions/clp/embedded/snaps/salto-conf-hardware$

But i am curious to know reason and if this can be fixed more elegant way @mr_cal ?

mr_cal · August 9, 2024, 12:47pm

It looks like it starts working after giving access to other users. It’s possible that the gitlab-runner user isn’t properly added to the lxd group. Since adding a user to a group normally requires a new session, I’ve seen a few different workarounds in CI. One is to run sg lxd -c "snapcraft". Another is to change the daemon group as is done here: setup-lxd/action.yml at 2aa6f7caf7d1484298a64192f7f63a6684e648a4 · canonical/setup-lxd · GitHub.