Local sources are not copied to $SNAPCRAFT_PART_SRC

I’m giving a go at upgrading one of my snaps from core18 to core20. One of the parts has a source: patches stanza, which copies patch files under that directory to the stage area to be later applied in another part’s build step. The original implementation (that works today with core18 EDIT: it works on launchpad builders that use LXD, but fails similarly locally) looks like this:

part-name:
  plugin: nil
  source: patches
  override-build: |
    mkdir -p $SNAPCRAFT_PART_INSTALL/patches
    cp *.patch $SNAPCRAFT_PART_INSTALL/patches/
  override-prime: ""

I also tried simplifying it to:

part-name:
  plugin: dump
  source: patches
  organize:
    '*.patch' : patches/

In both cases though, when running snapcraft (version 4.2, classic snap, revision 5407 from the stable channel) with the default multipass backend (version 1.4.0, revision 2445 from the stable channel), the build fails because the patches are not copied.

In the corresponding multipass VM:

snapcraft-0ad ../parts/0ad-patches# ls -r *
state:
stage  pull  build

src:

run:
build.sh

install:

build:

What am I missing?

override-prime: "" short circuits the prime actions and prevents anything from getting primed…

dropping the override-prime and the organize version both work for me.

Is patches by chance a symlink outside of the project?

I know, and that’s the intended behaviour: the patches are useful only for building, I don’t want them to end up in the snap.

No, it’s a plain directory containing two .patch files.

I simplified the part definition to this:

part-name:
  plugin: dump
  source: patches

and I’m still seeing /root/parts/part-name/{src,build,install} in the multipass VM are empty after (seemingly) successfully running the pull, build and stage phases for this part.

I’m puzzled.

Can you try a snapcraft clean and retry?

FWIW:

snapcraft-my-snap-name # find /root/parts/part-name/
/root/parts/part-name/
/root/parts/part-name/src
/root/parts/part-name/src/1.patch
/root/parts/part-name/run
/root/parts/part-name/run/build.sh
/root/parts/part-name/build
/root/parts/part-name/build/1.patch
/root/parts/part-name/state
/root/parts/part-name/state/stage
/root/parts/part-name/state/prime
/root/parts/part-name/state/build
/root/parts/part-name/state/pull
/root/parts/part-name/install
/root/parts/part-name/install/1.patch

If you can share your reproducer, that’d help too…

Lastly, if you still have problems after a clean, make sure you aren’t seeing any denials in the logs, the project is in $HOME, and multipass is up-to-date?

I should have started there, of course: https://code.launchpad.net/~osomon/+junk/0ad-snap-core20

snapcraft --use-lxd successfully builds the 0ad-patches part (it fails later on but it’s an unrelated issue). However with the default multipass backend, it consistently fails to copy the local sources (even after cleaning, or using multipass to delete the VM and have snapcraft re-create it from scratch).

I’m not seeing any denials in the logs, my project is in $HOME and multipass is up-to-date. One tweak to the default configuration is that my multipass VMs are stored on a separate disk, and that directory is bind-mounted to /var/snap/multipass. I don’t expect this would make a difference, but maybe this has subtle side effects?

Do you by any chance happened to encounter the following bugs?

This explains why the multipass build fails while the LXD and Launchpad builds are fine.

REF: Bug #1891851 “/root/project is Multipass mount inaccessible when the project dir path contains Chinese characters” : Bugs : Snapcraft

The directories I’ve been using in this context contain only ASCII characters, and no spaces. So that’s a different issue, it seems.

1 Like

I just tried snapcraft-0ad and it seemed to work fine (up until a missing build-package dependency).

does multipass exec snapcraft-0ad sudo cat /root/project/patches/use-sdl-semaphores.patch work?

Yes, it does print the patch file to the console.

Interestingly, if I shell into the VM and use sudo su to become root, /root/project/ appears empty. Is that expected?

Lastly, if you still have problems after a clean, make sure you aren’t seeing any denials in the logs, the project is in $HOME, and multipass is up-to-date?

Sorry - to jump on someone else’s question - I have equivalent failures (source missing from the src directory) - on the dump wrapper.

  • Can you direct me to links about “denials” and how to diagnose? I do indeed see apparmor=DENIED for snap.multipass.mulitipassd, but no context to diagnose it further.
  • Additionally you’ve highlighted multipass operations - is there a way to get snapcraft to dump out each step it is taking (equivalent to --debug or set -x) so as to be able to identify which exact step is not proceeding as expected.

If it helps start a new thread let me know

CC @Saviq, any thoughts?

Hi @whitty, sorry for the late reply, this got lost in the influx of email :slight_smile:

We do expect some to be logged still, even in normal Multipass operation. snappy-debug is a helpful debugging tool - if you show us your denials, we can tell you if it’s a real problem.

Exporting SNAPCRAFT_ENABLE_DEVELOPER_DEBUG=yes should help with this.

@Saviq, Thanks for your help - I eventually tracked it down to running from an encrypted home-directory. Is there somewhere I should log a bug - and if so -against snapcraft? or multipass?

1 Like

Fwiw, I hit the same issue, would be great if snapcraft/multipass could somehow detect that it’s running on encrypted home and show some warning or something.

There is a related bug report filed a while back: https://github.com/canonical/multipass/issues/1325#issuecomment-579843228