pycache is now added by default, using multiple parts means you probably want to inhibit cache generation or filter those out with filesets and stage.
Is there / could there be a plugin-specific keyword to inhibit the cache generation and so reverting to previous behaviour? This would help when maintain snaps across a number of bases.
This time, it might work if you can run find and remove all the cache files after snapcraftctl build but you shouldn’t expect the same language as new bases are introduced, the technology stack does evolve and we try to push the needle to what is current and best behavior at the time of introducing a new base.
It seems that there is a regression with the autotools plugin, it does not recognize “configflags” section:
$ snapcraft
Failed to load plugin: properties failed to load for networkmanager: Additional properties are not allowed ('configflags' was unexpected)
This is the snapcraft.yaml that failed:
This is not a regression for core20, as stated in the release notes, this flag has been renamed for consistency for core20 plugins.
This might be related, as I’m trying to use snapcraft 4.0 with LXD to build on ARM64.
Is it expected that snapcraft cannot launch a core20-compatible container or vm for building a snap (using lxd or multipass respectively)?
$ snapcraft
Launching a VM.
launch failed: Unable to find an image matching "core20"
An error occurred with the instance when trying to launch with 'multipass': returned exit code 2.
Ensure that 'multipass' is setup correctly and try again.
edit: oh, reproducing the lxd variant to capture the log output to post here, it seems to have successfully started a container this time around…
This is not a regression for core20, as stated in the release notes, this flag has been renamed for consistency for core20 plugins
Why is base: being used for versioning? Would not be better to introduce a version: section? Because base can in theory be different from coreXX, and what it specifies is orthogonal to the yaml spec version.
Also, some better error message would help here.
It’s not without precedent though. The last time major incompatible changes were made to Snapcraft, it was conditional on the base: keyword.
Granted, you could get access to all the new features with the old base then by adding base: core to your project. But if you wanted to use the new core18 base, you also needed to update your project to be compatible with the new Snapcraft. The same is true here for core20.
I used stage to remove all the files that collide, but now get the following traceback from snapcraft:
Traceback (most recent call last):
File "/snap/snapcraft/4657/bin/snapcraft", line 11, in <module>
load_entry_point('snapcraft==3.9.1', 'console_scripts', 'snapcraft')()
File "/snap/snapcraft/4657/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/click/core.py", line 1236, in invoke
return Command.invoke(self, ctx)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/cli/_runner.py", line 102, in run
snap_command.invoke(ctx)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/cli/_command.py", line 88, in invoke
return super().invoke(ctx)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/cli/lifecycle.py", line 272, in snap
_execute(steps.PRIME, parts=tuple(), pack_project=True, output=output, **kwargs)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/cli/lifecycle.py", line 77, in _execute
lifecycle.execute(step, project_config, parts)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 134, in execute
executor.run(step, part_names)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 188, in run
self._handle_step(part_names, part, step, current_step, cli_config)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 202, in _handle_step
getattr(self, "_run_{}".format(current_step.name))(part)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 244, in _run_pull
self._run_step(step=steps.PULL, part=part, progress="Pulling")
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 319, in _run_step
self._prepare_step(step=step, part=part)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 313, in _prepare_step
common.env = self.parts_config.build_env_for_part(part)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/internal/project_loader/_parts_config.py", line 279, in build_env_for_part
env += dep_part.env(stagedir)
File "/snap/snapcraft/4657/lib/python3.6/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 1044, in env
return self.plugin.env(root)
AttributeError: 'PythonPlugin' object has no attribute 'env'
Where does this need to be fixed?
Update:
I think this is because I’m trying to use a local plugin to handle parts of type plainbox-provider. At the moment this plugin is based on PluginV1, but maybe this isn’t a good idea. Should I consider trying to make a PluginV2 version of plainbox-provider?
Correct, we do not version plugins. Once a plugin is out for a given base, we support it for the lifetime of the base.
Please log a bug with the error message you got and what you would like to see, and we will look into it.
It could work, was this AttributeError sent to sentry?
There could be some interesting interactions between what a V1 plugin expects with regards to the environment, but other than that, for a local plugin should be acceptable.
4.0.1 is now on candidate with bug fixes, some of which were reported by you whom we thank for testing 4.0.
1 Like
I am sad =(
core20 itself, fails to be built with snapcraft candidate. See
https://launchpadlibrarian.net/479470444/buildlog_snap_ubuntu_focal_amd64_test-base-build-base-core20_BUILDING.txt.gz
I think we do weird stuff (cause host might not match chroot, so we hack build environment to execute things from a chroot) and it feels like the environment variables we rely on, are no there anymore and/or stuff moved.
Staging bootstrap
Priming bootstrap
make: *** No rule to make target 'test'. Stop.
Failed to run 'override-prime': Exit code was 2.
Build failed
Cause if you look at https://github.com/snapcore/core20/blob/master/snapcraft.yaml#L30
we do
override-prime: |
unset LD_LIBRARY_PATH;
export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin"
# ensure snapcraftctl is found, see
# https://github.com/snapcore/snapcraft/pull/2251
export PATH="$PATH:/snap/snapcraft/current/bin/scriptlet-bin"
snapcraftctl prime
# ensure build-in tests are run
cd ${SNAPCRAFT_PART_SRC} && make test TESTDIR=${SNAPCRAFT_PRIME}
And it feels like SNAPCRAFT_PART_SRC is no longer set. At override-prime step.
Now. what we do is weird, but i think the intention is to setup a chroot of what will be core20 (SNAPCRAFT_PRIME), and execute tests from source-tree against it. And I can’t find my source-tree anymore (which has makefile, with the test target, which knows how to execute them against a chroot).
How should I be doing this?
Wouldn’t it be easier to run your tests from override-build? Something like:
override-build: |
# Do whatever environment hacks you need here, but maybe try without them first?
snapcraftctl build
make test TESTDIR=$SNAPCRAFT_PART_INSTALL
You’ve only got a single part, and aren’t filtering out any files for the stage or prime steps, so this should be equivalent. You could probably go without the override-stage and override-prime sections then.
1 Like
@sergiusens so, you are saying that plugin behavior is tied to a given base snap? I am fine with that, but I have not seen that clearly stated previously.
This problem is also affecting some of our snaps, we use SNAPCRAFT_PART_SRC and SNAPCRAFT_PROJECT_DIR in override-prime.
This statement has been made in the release notes of 3.0, 4.0 and in the docs.
I would like to know more about this use case for SNAPCRAFT_PART_SRC to be available during prime.
That said, SNAPCRAFT_PROJECT_DIR should be exported for stage and prime.