Specifying architecture on cross-compilations

while i can define “on” in stage-packages to get different arch specific packages installed when building for different target arches, this is not supported for build-packages …

i.e:

stage-packages:
  - foobar on amd64

works … but:

build-packages:
  - foobar on amd64

…is currently not supported

it would be good if the two could be consistent … typically when cross building for armhf you want the gcc-arm-linux-gnueabihf installed via build-packages, but this package only exists on non-armhf architectures … so if you would just blantly put it into the build-packages list, the package wouldnt be buildable natively anymore … supporting the above would fix this …

2 Likes

If you use --target-arch the package would be automatically added to build-packages. As of recently, the same is true for libc6-dev-armhf-cross (and analogues). So there shouldn’t be a case where you have to specify these manually. Although you would still run into the problem with other packages that aren’t special-cased like this. Maybe you could add a more concrete example.

my actual end-goal for all my snaps is that you can just run snapcraft everywhere without any arch specific special treatment like defining an additional cmdline option…

first of all i think the *-packages directives should be consistent in their options and behaviour…

also, if i use the architectures keyword in my snapcraft.yaml and the target architecture defined there is not matching my host arch, why cant snapcraft install the compiler automatically instead of requiring me to use --target-arch.

especially for gadgets and kernels you typically have exactly only one target arch in your yaml and will not use additional libs, in these cases gcc-${target arch} should simply be installed automatically without needing an additional option.

i dont really have more complex examples, my main focus is to get the most porter friendly setup working in our reference gadgets and for these packages i dont need additional foreign libs typically, so here such an automation would be enough without enabling the “on” keyword (still leaving the inconsistency between build-packages and stage-packages though)…

There’s a separate discussion on changing the architectures field.

I’m not disagreeing at this point, but I’d like to figure out the best solution for the case at hand and not looking into a new feature to work around a bug. That’s also why I was asking for other examples where this would be useful.

To be honest, the proposal above by @ogra feels like fixing a bug rather than working around one, and that bug is that stage-packages and build-packages do not work the same way. The expectation that they do seems sane and will definitely be implied by people just based on the field names and their similar content.

I also agree with @ogra that changing what gets installed implicitly based on whether or not --target-arch was used sounds bogus if the actual result is the same. In other words, if the snap for arm is going to be built, the behavior should not change whether I said --target-arch=arm or not.

on for snapcraft primitives is on the roadmap, we had already discussed this with @kyrofa at our mini sprint and made notes of it.

It sounds like you are talking about something more general. @ogra is asking to just make stage-packages and build-packages work the same.

Speaking of which, is the current syntax the stage-packages the one discussed many months ago, or has that change (or not been implemented yet)?

It is the same one from back then, it is implemented (you reviewed the PR :slight_smile: ) and it is already being used.

@ogra among many others have requested to extend this functionality for source (useful when using dump to specify tarballs per architecture) entries and build-packages (which I actually have a use case for right now).

1 Like

I’m moving @kalikiana’s related topic here as the subject is the same.

Some context

  • When cross-compiling Snapcraft implicity installs architecture-specific packages for gcc and libc, internally called cross-build-packages, for example gcc-arm-linux-gnueabihf and libc6-dev-armhf-cross when building for armhf on a different architecture.
  • There’s a discussion under way to support on selectors in build-packages to list different packages conditionally which only exist for a specific architecture.

Problem: packages for the target architecture

What’s currently missing is support for multi-arch packages installed for the target architecture rather than the host architecture. When cross-compiling for example from amd64 to armhf build-packages: [libglib2.0-dev] will install amd64 binaries that can’t be linked to an armhf target binary. What’s actually required is the same package, but built for the target architecture, armhf. Snapcraft needs to learn how to transparently install packages for the target architecture.

Proposal

Get build-packages for the correct architecture.

Note: no changes to YAML or plugin API involved.

Implementation: leveraging multi-arch internally

I started working on this some time ago with the purpose of using C libraries from build-packages from Go. But it’s proven difficult to review so this post is meant to facilitate discussion of the implementation. Note that all of the following is Debian-specific: support for multi-arch on Fedora would use different package names, different syntax, different host setup and other tools, behind the scenes.

  • in Debian multi-arch syntax :armhf specifies a package for the target-architecture, :native specifies a package from the host architecture - this is very simplified, but basically these need to be added internally
  • apt-get build-dep can be used, provided the host has source packages configured, to figure out the multi-arch equivalent of a set of packages - it’s shown to be difficult to handle as a tool because errors can be ambiguous and difficult to parse (including incompatible syntax of the form libpkg2=3:0.4-0ubu5:armhf rather than libpkg2:armhf=3:0.4-0ubu as understood by dpkg/ apt)
  • Python APT bindings understand arch suffixes but have no equivalent of I need libglib2.0-dev for cross-compiling

To make any of this work the host needs to be setup beforehand, which in the case of cleanbuild can’t be done manually.

  • The architecture needs to be registered via dpkg --add-architecture armhf
  • /etc/apt/sources.list* requires package sources with [arch=armhf] to pull in foreign-arch packages
1 Like

How’s that different from the thread to support “on” in build packages? I thought the whole point @ogra was raising there was precisely to support the existing syntax for multiple architectures in that field. If so, let’s please unify the conversations again.

@sergiusens Do you have some pointers for our prior discussion on how the “on” syntax works? Seems quite relevant here, and some of the points above don’t seem entirely aligned with it (first time I hear of :native for example). I’m also a bit worried about the fact the message above is not considering other packaging systems at all.

@kalikiana please don’t mix these two concepts and please be more precise in telling the story about why this is needed so there is no confusion with what @ogra raised.

:native is purely part of deb/apt semantics, but I’ll let @kalikiana clean up this proposal himself.

I think the failure here was a clear separation in the conversation between what to do when cross compiling and the arch/distro specific selectors. The latter is documented here: https://snapcraft.io/docs/build-snaps/syntax (there is no anchor so search for the stage-packages bullet)

Yes, I’m pretty lost now. We have way too many topics talking about the exact same subject.

@kalikiana, given my previous feedback and now @niemeyer’s can you work on more precise subjects as well?

I updated the post to more clearly separate context, problem description, proposal and implementation. Please let me know if there’s more need for clarification.

It’s a bit more clear, but I’m still missing something.

When you say “apt-get build-dep can be used, provided the host has source packages configured, to figure out the multi-arch equivalent of a set of packages”, that sounds to me like magic which we don’t want. We have an explicit syntax defining which packages should be installed on which architectures. All we want, I believe, is to be able to use the existing “on” syntax which @ogra asked for and which is already being used in snapcraft.yaml in stage-packages.

Also, as a minor note, we’re not adding the support for “:native” in snapcraft.yaml, right? That sounds implied and not very useful.

@sergiusens Can you please organize a call with the three of us, plus @ogra and @mvo? We’ve been discussing this for a bit too long without much clarity, so a call will probably help sorting out these details more easily.

Done. I was planning on creating an alternate post going over what each thing is in a “clearing the air” type of prose, I might do it anyways, just wondering now if I should do it after the call.

1 Like

We need less posts on this topic, rather than more. Part of the issue is precisely that we have so many concurrent conversations about what is apparently the same thing.

We just had a very nice call on this topic (thanks for organizing @sergiusens) and the agreements feel quite promising. In a nutshell:

  1. The existing “on <arch>:” syntax will be accepted under build-packages as well
  2. The default architecture for packages under “on <arch>” is the one from the build machine
  3. Packages under “on <arch>:” are installed only when the build machine has that architecture
  4. A new “to <arch>:” statement will be supported in build-packages and stage-packages
  5. The default architecture for packages under “to <arch>:” is the target architecture
  6. Packages under “to <arch>:” are installed only when the target architecture is the one listed, which includes both a cross-compilation towards that architecture and a native build on that architecture
  7. The two statements may be used together as “on <arch> to <arch>:” to precisely specify the case of a particular cross-compilation
  8. The default architecture for packages under “on <arch> to <arch>:” is the target architecture
  9. The default architecture for top-level packages (those outside of any “on” or “to” sections) is the one from the build machine
  10. The following suffixes may be used after any package name under stage-packages and build-packages to override the default architecture:
    10.1. “:native” - package must have architecture of the build machine
    10.2. “:target” - package must have the target architecture
    10.3. “:<arch>” - package must have the specified architecture
3 Likes