Call for testing: Snapcraft 7.2.0

jocado · October 27, 2022, 3:50pm

Cross posting - this seems relevant: Snapcraft 7.2 depends on snapd socket at runtime

Cheers, Just

mr_cal · October 27, 2022, 11:21pm

@EndrII, would you be so kind to test my fix?

You can install it with snap refresh snapcraft --channel=latest/edge/pr-3965

I believe the issue is related to a limitation with Multipass snap only having access to the user’s home directory due to snap confinement.

mr_cal · October 27, 2022, 11:23pm

Thanks for the report, @jocado. I’ll look into that tomorrow.

EndrII · October 28, 2022, 8:39am

Yes, after your fixes, Snapcraft works great. Thank you

gward-tridium · October 31, 2022, 11:45am

Attempting to run any snap builds in Docker (using https://github.com/snapcore/snapcraft/blob/main/docker/Dockerfile as a base) fails with Snapcraft 7.2.3:

Traceback (most recent call last):
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/cli.py", line 228, in run
    _run_dispatcher(dispatcher)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/cli.py", line 204, in _run_dispatcher
    dispatcher.run()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/craft_cli/dispatcher.py", line 448, in run
    return self._loaded_command.run(self._parsed_command_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/commands/lifecycle.py", line 259, in run
    super().run(parsed_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/commands/lifecycle.py", line 132, in run
    parts_lifecycle.run(self.name, parsed_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 172, in run
    yaml_data = process_yaml(snap_project.project_file)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 135, in process_yaml
    yaml_data = yaml_utils.load(yaml_file)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/yaml_utils.py", line 92, in load
    raise errors.LegacyFallback("base is not core22")
snapcraft.errors.LegacyFallback: base is not core22

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snaphelpers/_env.py", line 58, in __getattr__
    return self._env[attr]
KeyError: 'INSTANCE_NAME'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/legacy.py", line 27, in legacy_run
    run()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/_runner.py", line 130, in run
    snap_command.invoke(ctx)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/lifecycle.py", line 406, in snap
    _execute(steps.PRIME, parts=tuple(), pack_project=True, output=output, **kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/lifecycle.py", line 77, in _execute
    build_provider = get_build_provider(**kwargs)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/cli/_options.py", line 304, in get_build_provider
    snap_config = common.get_snap_config()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft_legacy/internal/common.py", line 390, in get_snap_config
    snap_config = SnapConfigOptions(keys=["provider"])
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snaphelpers/_conf.py", line 47, in __init__
    self._snapctl = snapctl or SnapCtl()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snaphelpers/_ctl.py", line 68, in __init__
    self._instance_name = env.INSTANCE_NAME
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snaphelpers/_env.py", line 60, in __getattr__
    raise AttributeError(str(e))
AttributeError: 'INSTANCE_NAME'

I came up with a workaround with some hacking around in snapcraft_legacy/internal/common.py:

diff --git a/snapcraft_legacy/internal/common.py b/snapcraft_legacy/internal/common.py
index ad6b0949..c69a5c51 100644
--- a/snapcraft_legacy/internal/common.py
+++ b/snapcraft_legacy/internal/common.py
@@ -384,6 +384,12 @@ def get_snap_config() -> Optional[Dict[str, str]]:
         )
         return None

+    if is_process_container():
+        logger.debug(
+            "Not loading snap config because snapcraft is running in a container"
+        )
+        return None
+
     snap_config = SnapConfigOptions(keys=["provider"])
     snap_config.fetch()

which at least allows the snapcraft commands to complete successfully if I modify that file locally.

mr_cal · October 31, 2022, 12:48pm

Thanks @gward-tridium. @jocado reported the same issue here. I’m working on a fix.

mr_cal · November 1, 2022, 8:25am

@EndrII - Your issue with running snapcraft + multipass is now fixed and released on the channel latest/candidate

EndrII · November 1, 2022, 8:36am

Thank you, but I will wait for stable release )

mr_cal · November 2, 2022, 10:05am

And 7.2.4 is on stable!

andrewsomething · November 2, 2022, 9:51pm

We’re seeing the same issue as reported by gward-tridium with Snapcraft v7.2.4.

Let me know if there’s anything we can do to help debug.

mr_cal · November 3, 2022, 9:44am

@andrewsomething, I have a fix here, would you be able to test it using channel=edge/pr-3971?

andrewsomething · November 3, 2022, 1:35pm

@mr_cal I can confirm that resolves the issue for me. Thanks for the quick work!

gward-tridium · November 3, 2022, 1:58pm

@mr_cal The fix on edge/pr-3971 fixes our issues as well. Thanks for taking a look!

mr_cal · November 3, 2022, 2:29pm

Great! Thank you @gward-tridium, @andrewsomething, and @jocado for the good feedback and testing. This fix is now released to the channel latest/candidate

jocado · November 3, 2022, 3:58pm

Great - thank you for your support

hardeves94 · November 3, 2022, 9:43pm

Just used channel latest/candidate, but still have the issue ‘“http://localhost/v2/snapctl”: dial unix /run/snapd-snap.socket: connect: no such file or directory’

Trace:

 Traceback (most recent call last):
  File "/snap/snapcraft/current/bin/snapcraft", line 8, in <module>
    sys.exit(run())
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/cli.py", line 228, in run
    _run_dispatcher(dispatcher)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/cli.py", line 204, in _run_dispatcher
    dispatcher.run()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/craft_cli/dispatcher.py", line 448, in run
    return self._loaded_command.run(self._parsed_command_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/commands/lifecycle.py", line 259, in run
    super().run(parsed_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/commands/lifecycle.py", line 132, in run
    parts_lifecycle.run(self.name, parsed_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 208, in run
    _run_command(
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 249, in _run_command
    _run_in_provider(project, command_name, parsed_args)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 473, in _run_in_provider
    provider = providers.get_provider(provider_name)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/providers.py", line 210, in get_provider
    snap_config = get_snap_config()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/snap_config.py", line 88, in get_snap_config
    snap_config.fetch()
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snaphelpers/_conf.py", line 78, in fetch
    self._config = self._snapctl.config_get(*self._keys)
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snaphelpers/_ctl.py", line 134, in config_get
    conf = json.loads(self.run("get", "-d", *keys))
  File "/snap/snapcraft/current/lib/python3.8/site-packages/snaphelpers/_ctl.py", line 175, in run
    raise SnapCtlError(process)
snaphelpers._ctl.SnapCtlError: Call to snapctl failed with error 1: error: cannot communicate with server: Post "http://localhost/v2/snapctl": dial unix /run/snapd-snap.socket: connect: no such file or directory

Was I too soon in using latest/candidate?

Ah wait, might be caused by me using jammy as UBUNTU arg?

mr_cal · November 4, 2022, 9:35am

UBUNTU=Jammy is not the problem. From your logs, the first call to snapctl works but the second does not. Curious behavior, I can’t reproduce that or figure out how it happened.

Regardless, the fix is simple and I’ll have prepared today.

Can you provide some details on how you are doing setup, execution, or provide the snapcraft.yaml? It looks like a core22 snap being built non-interactively. I’d like to write regression tests to not break docker builds again.

Update 1: PR is here and can be tested on channel edge/pr-3974

Update 2: Released as version 7.2.7 and should be available on the channel candidate soon

hardeves94 · November 4, 2022, 3:35pm

I will test it asap. I did adapt the Dockerfile as well, added core22 to it.

My yml file:

name: clever # you probably want to 'snapcraft register ’ base: core22 # the base snap is the execution environment for this snap version: ‘master’ # just for humans, typically ‘1.2+git’ or ‘1.3.2’ summary: Clever PSI/SI generation description: | Linear broadcast services require Program Specific Information/System Information (PSI/SI) as an essential part of the stream. Tuning information as well as Electronic Program Guide (EPG) can be generated by the Dualz “CLEVER” PSI/SI Generator. The Dualz PSI/SI Generator conforms MPEG and DVB standards. Generated tables include PAT, PMT, CAT, BAT, NIT, SDT, EIT pf/scheduled and more. Dualz PSI/SI Generator imports data from several interfaces, from file (XML) or REST API interfaces grade: devel # must be ‘stable’ to release into candidate/stable channels confinement: devmode # use ‘strict’ once you have the right plugs and slots

parts: clever: plugin: dump source: ./clever stage-packages: - openssl #organize: # ‘clever/CyclerManager/CyclerApp’: usr/bin/CyclerApp

install-hook: plugin: dump source: install-hook/ organize: install-config: snap/hooks/install

zixi: plugin: dump source: ./zixi_linux.tar.gz organize: ‘Linux/.so’ : usr/lib/ stage: [ usr/lib/libzixiFeeder.so, usr/lib/libzixiClient.so ]

locales-launch: source: https://github.com/Lin-Buo-Ren/locales-launch.git source-tag: v1.0.0 plugin: dump stage-packages: # For localedef(1) - libc-bin # For locale data used by localedef(1) - locales # All pre-generated locale data # DISABLED: Lots of duplicate data will be included in snap when this is staged, unless the target machine has no resource to generate them in runtime otherwise not using it # # Lack of compiled locales breaks gettext based localisation - snapcraft - snapcraft.io # Lack of compiled locales breaks gettext based localisation # # - locales-all

stage:
  - bin/locales-launch
  - etc/locale.alias
  - usr/bin/localedef
  - usr/share/doc/locales
  - usr/share/i18n
  - usr/share/locale

apps: cleverengine: command: CleverEngine/CleverEngine $SNAP_COMMON/cleverengine.conf daemon: simple plugs: - network - network-bind

cyclermanager: command: CyclerManager/CyclerManager $SNAP_COMMON/cyclermanager.conf daemon: simple plugs: - network - network-bind

CyclerApp: command: CyclerManager/CyclerApp plugs: - network - network-bind

psisiinjector: command-chain: [bin/locales-launch] command: PSISIInjector/PSISIInjector $SNAP_COMMON/injector.conf daemon: simple plugs: - network - network-bind

epgparser: command-chain: [bin/locales-launch] command: EPGParser/EPGParser $SNAP_COMMON/epgparser.conf daemon: simple plugs: - network - network-bind

layout: /var/log/clever: bind: $SNAP_DATA/var/log/clever /var/local/clever: bind: $SNAP_DATA/var/local/clever /var/tmp: bind: $SNAP_DATA/var/tmp

I hope it is readable, not sure how to add code to comment…

hardeves94 · November 4, 2022, 4:01pm

Different error: Traceback (most recent call last): File “/snap/snapcraft/current/bin/snapcraft”, line 8, in sys.exit(run()) File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/cli.py”, line 228, in run _run_dispatcher(dispatcher) File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/cli.py”, line 204, in _run_dispatcher dispatcher.run() File “/snap/snapcraft/current/lib/python3.8/site-packages/craft_cli/dispatcher.py”, line 448, in run return self._loaded_command.run(self._parsed_command_args) File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/commands/lifecycle.py”, line 259, in run super().run(parsed_args) File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/commands/lifecycle.py”, line 132, in run parts_lifecycle.run(self.name, parsed_args) File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py”, line 208, in run _run_command( File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py”, line 249, in _run_command _run_in_provider(project, command_name, parsed_args) File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py”, line 475, in _run_in_provider providers.ensure_provider_is_available(provider) File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/providers.py”, line 78, in ensure_provider_is_available if not LXDProvider.is_provider_installed() and not confirm_with_user( File “/snap/snapcraft/current/lib/python3.8/site-packages/snapcraft/utils.py”, line 259, in confirm_with_user with emit.pause(): File “/snap/snapcraft/current/lib/python3.8/site-packages/craft_cli/messages.py”, line 694, in func raise RuntimeError(“Emitter is stopped already”) RuntimeError: Emitter is stopped already

I did adapt the dockerfile, and added core22 to it

mr_cal · November 10, 2022, 9:41pm

Thanks @hardeves94! Your subsequent error should be fixed shortly with https://github.com/snapcore/snapcraft/pull/3982.

P.S. You can add code blocks by adding a line with 3 backticks (`) before and after the code.