Issues dockerizing a snapcraft build process

ztjoseph · June 1, 2022, 4:32pm

Hello.

I’ve recently tried following the instructions for running my snap’s build process in a Docker container but have run into a few problems and they do not appear to be covered in the docs and I’m suspecting the Dockerfile may be out of date?

I’ve tried the Dockerfile here

But I end up with the following error:

The following snaps are required but not installed as snapcraft is running inside docker or podman container: core18.
Please ensure the environment is properly setup before continuing.                                                                                                                                                                       
Ignore this message if the appropriate measures have already been taken                                                                                                                                                                  
Pulling one 
+ snapcraftctl pull
Sorry, an error occurred in Snapcraft:
[Errno 22] Invalid argument: '/sys/kernel/mm/page_idle/bitmap' -> '/build/parts/one/src/sys/kernel/mm/page_idle/bitmap'

I then attempt to install core18 in the container using sudo snap install core18 but then I notice that the snapd daemon appears to be completely unresponsive. Things like snap list will never execute. When I attempt to start the snapd process I’m presented with an error stating that the container wasn’t started with systemd.

Are there more up to date docs/Dockerfiles somewhere?

I’m trying to build a pretty simple snap and I’m continually running into issues in every direction following the docs. Can someone take a look at this?

Thanks!

dmourn · June 1, 2022, 9:13pm

I don’t think it has ever been possible to access snap in that way from inside a docker container.

I could be wrong, but I think the idea is to use the docker container as your build environment in place of the core snaps you typically pull in. Hence the, Ignore this message if the appropriate measures have already been taken which implies you might have to make some changes.

For example, the snap I’m working on I’m using the gnome-3-38 extension. To get the docker image to work I had to add a lot of packages to my stage-packages. I ended up with wayyyyy more errors than I started but they were all at the prime step and the finale snap was functional.

Although, I never encountered anything like your final error. Can you share the snapcraft.yaml? It may be enlightening

ztjoseph · June 2, 2022, 1:32am

Thanks. This helps clarify somewhat but it still doesn’t seem to work. If I’m interpreting you correctly I likely don’t need to specify a base: core18 in my yaml file since the Dockerfile sets all of that up. However when I remove this requirement and use base: bare I get the exact same error. If I remove base: entirely I get a parsing error (makes sense I guess).

This is a standard snapcraft.yaml file that works just fine locally so I doubt the issue is with that.

EDIT: Here’s the backtrace of the error mentioned previously. I haven’t changed anything in the dockerfile and am following the instructions verbatim. But first, of particular interest is this excerpt from the backtrace:

    raise errors.LegacyFallback("base is not core22")
snapcraft.errors.LegacyFallback: base is not core22

Why would it complain of this?

 + snapcraftctl pull
Sorry, an error occurred in Snapcraft:
[Errno 22] Invalid argument: '/sys/kernel/mm/page_idle/bitmap' -> '/build/parts/one/src/sys/kernel/mm/page_idle/bitmap'
We would appreciate it if you anonymously reported this issue.
No other data than the traceback and the version of snapcraft in use will be sent.
Would you like to send this error data? (Yes/No/Always/View) [no]: View
Traceback (most recent call last):

  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/cli.py", line 161, in run
    dispatcher.run()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/craft_cli/dispatcher.py", line 406, in run
    return self._loaded_command.run(self._parsed_command_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/commands/lifecycle.py", line 206, in run
    super().run(parsed_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/commands/lifecycle.py", line 82, in run
    parts_lifecycle.run(self.name, parsed_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 134, in run
    yaml_data = process_yaml(snap_project.project_file)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 85, in process_yaml
    yaml_data = yaml_utils.load(yaml_file)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/yaml_utils.py", line 92, in load
    raise errors.LegacyFallback("base is not core22")
snapcraft.errors.LegacyFallback: base is not core22

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/file_utils.py", line 102, in link_or_copy
    link(source, destination, follow_symlinks=follow_symlinks)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/file_utils.py", line 136, in link
    os.link(source_path, destination, follow_symlinks=False)
OSError: [Errno 18] Invalid cross-device link: '/sys/kernel/mm/page_idle/bitmap' -> '/build/parts/one/src/sys/kernel/mm/page_idle/bitmap'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/snapcraft/current/bin/snapcraft", line 8, in <module>
    sys.exit(run())
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/cli.py", line 185, in run
    legacy.legacy_run()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/legacy.py", line 25, in legacy_run
    run()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/_runner.py", line 130, in run
    snap_command.invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/lifecycle.py", line 387, in snap
    _execute(steps.PRIME, parts=tuple(), pack_project=True, output=output, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/lifecycle.py", line 102, in _execute
    lifecycle.execute(step, project_config, parts)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/lifecycle/_runner.py", line 145, in execute
    executor.run(step, part_names)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/lifecycle/_runner.py", line 199, in run
    self._handle_step(part_names, part, step, current_step, cli_config)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/lifecycle/_runner.py", line 213, in _handle_step
    getattr(self, "_run_{}".format(current_step.name))(part)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/lifecycle/_runner.py", line 255, in _run_pull
    self._run_step(step=steps.PULL, part=part, progress="Pulling")
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/lifecycle/_runner.py", line 356, in _run_step
    getattr(part, step.name)()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/pluginhandler/__init__.py", line 502, in pull
    self._do_runner_step(steps.PULL)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/pluginhandler/__init__.py", line 292, in _do_runner_step
    return getattr(self._runner, "{}".format(step.name))()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/pluginhandler/_runner.py", line 81, in pull
    self._run_scriptlet(
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/pluginhandler/_runner.py", line 189, in _run_scriptlet
    raise error
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/pluginhandler/_runner.py", line 178, in _run_scriptlet
    self._handle_builtin_function(
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/pluginhandler/_runner.py", line 233, in _handle_builtin_function
    function(**function_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/pluginhandler/__init__.py", line 522, in _do_pull
    self.source_handler.pull()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/sources/_local.py", line 37, in pull
    file_utils.link_or_copy_tree(
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/file_utils.py", line 245, in link_or_copy_tree
    copy_function(source, destination)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/file_utils.py", line 110, in link_or_copy
    copy(source, destination, follow_symlinks=follow_symlinks)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/file_utils.py", line 159, in copy
    shutil.copy2(source, destination, follow_symlinks=follow_symlinks)
  File "/snap/snapcraft/current/usr/lib/python3.8/shutil.py", line 435, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/snap/snapcraft/current/usr/lib/python3.8/shutil.py", line 275, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/snap/snapcraft/current/usr/lib/python3.8/shutil.py", line 172, in _fastcopy_sendfile
    raise err
  File "/snap/snapcraft/current/usr/lib/python3.8/shutil.py", line 152, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 22] Invalid argument: '/sys/kernel/mm/page_idle/bitmap' -> '/build/parts/one/src/sys/kernel/mm/page_idle/bitmap'

Changing my base to core22 out of desperation to get anything to work just triggers the following error which seems irrelevant:

Bad snapcraft.yaml content:
- extra field 'filesets' not permitted in 'parts.one' configuration

ztjoseph · June 2, 2022, 3:44am

Forgot to link to the snapcraft.yaml in question:

github.com

zerotier/ZeroTierOne/blob/1.8.7/snap/snapcraft.yaml

name: zerotier
summary: Securely connect any device, anywhere.
description: |

  ZeroTier is a software-based managed Ethernet switch for planet Earth. Use it to connect your
  desktop clients, servers, phones, NAS, or even individual applications (using our SDK).

  This snap contains ZeroTier One, a service that provides ZeroTier network connectivity and
  makes joining virtual networks as easy as joining IRC or Slack channels. Apps for Android
  and iOS are available for free in the Google Play and Apple app stores.

  ZeroTier eliminates the LAN/WAN distinction and makes VPNs, tunnels, proxies, and other kludges
  arising from the inflexible nature of physical networks obsolete. Everything is encrypted
  end-to-end and traffic takes the most direct (peer to peer) path available.

  Install (be sure to use sudo)

    sudo snap install zerotier

  Join your network

This file has been truncated. show original

cmatsuoka · June 20, 2022, 7:15pm

When you build using the built-in lxd containers, a suitable build environment that matches your base: setting is automatically used. If you run in destructive mode, you’re responsible for providing the environment that matches your base: configuration. That means that if your snap has base: core18 you need to use a bionic docker image, if it’s base: core20 then you need to use a focal docker image, and if you use base: core22 you need a jammy docker image. If you use a mismatched environment the result is unpredictable, you can have errors during the build and most likely the resulting snap will not work.

mohamin101 · January 2, 2023, 12:52pm

Hello @ztjoseph, i have the same issue you’ve had, so I wanted to know if you somehow figured out how to solve it?