Cross compiling packages for ubuntu core

snapcraft
1

Hi!

When developing software for embedded systems it is very useful to be able to cross compile the software on the developers workstation or build server. I know that snapcraft can build packages for arm devices on arm hardware (or virtual arm hardware such as qemu).

The question is:
Is it possible to cross compile c/c++ software (thats what i have been trying without success) on na intel laptop with armhf as target. At least the cmake plugin says:

snapcraft build --target-arch=armhf                
Setting target machine to 'armhf'
Traceback (most recent call last):
  File "/usr/bin/snapcraft", line 11, in <module>
    load_entry_point('snapcraft==2.34+17.10', 'console_scripts', 'snapcraft')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 564, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2662, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2316, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2322, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/snapcraft/cli/__main__.py", line 19, in <module>
    run(prog_name='snapcraft')
  File "/usr/lib/python3/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 697, in 
main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/snapcraft/cli/lifecycle.py", line 79, in build
    _execute('build', parts, **kwargs)
   File "/usr/lib/python3/dist-packages/snapcraft/cli/lifecycle.py", line 31, in _execute
    lifecycle.execute(command, project_options, parts)
   File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle.py", line 113, in execute
    config = snapcraft.internal.load_config(project_options)
   File "/usr/lib/python3/dist-packages/snapcraft/internal/project_loader/__init__.py", line 24, in load_config
    return Config(project_options)
   File "/usr/lib/python3/dist-packages/snapcraft/internal/project_loader/_config.py", line 124, in 
 __init__
snapcraft_yaml=self.snapcraft_yaml_path)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/project_loader/_parts_config.py", line 55, in __init__
    self._process_parts()
   File "/usr/lib/python3/dist-packages/snapcraft/internal/project_loader/_parts_config.py", line 85, in _process_parts
    self.load_part(part_name, plugin_name, properties)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/project_loader/_parts_config.py", line 169, in load_part
    definitions_schema=self._validator.definitions_schema)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/pluginhandler/_plugin_loader.py", line 78, in load_plugin
    plugin.enable_cross_compilation()
  File "/usr/lib/python3/dist-packages/snapcraft/_baseplugin.py", line 182, in enable_cross_compilation
'to a different target architecture'.format(self.name))

NotImplementedError: The plugin used by 'snapcraft-test-src' does not support cross-compiling to a different target architecture

My snapcraft.yaml file looks like this:

name: snapcraft-test-pkg 
version: '0.0.1' 
summary: A test package  
description: |
  The only purpose of the package is to test the snap build chain.

architectures:
        - armhf
        - x86_64

grade: devel 
confinement: strict 
apps:
        snap-test-exec: 
                command: snap-test-exec
parts:
        snapcraft-test-src:
                plugin: cmake
                configflags: 
                        - -DCMAKE_INSTALL_PREFIX=/usr

                source-type: git
                source: https://github.com/malo-rte/snapcraft-test-src.git
                build-packages: 
                        - build-essential
                        - gcc-arm-linux-gnueabihf 

                prime:
                        - usr/bin

Any help or suggestions is appreciated.

Best regards,
Mats Loman

1 Like
2

Hi @malo-rte
cmake pluging does not support cross compilation by default, however for your simple use case you can enable it by simply using own custom make plugin.
Just place following into: snap/plugins/x-cmake.py

import snapcraft
import logging

from snapcraft.plugins import cmake
logger = logging.getLogger(__name__)

class CMakePlugin(snapcraft.plugins.cmake.CMakePlugin):

    def enable_cross_compilation(self):
        logger.info('Cross compiling to {!r}'.format(self.project.kernel_arch))

    def _build_environment(self):
        env = super()._build_environment()
        if self.project.kernel_arch == 'arm':
            env['CC'] = 'arm-linux-gnueabihf-gcc'
            env['CXX'] = 'arm-linux-gnueabihf-g++'

        return env
3

while actual cross compilation with dependencies is pretty hard (unless you compile each dependent lib from a part in a cross manner), you can easily hack together something via snapcraft shell scriptlets for builds that do not have actual large dependency chains … i.e. see:

https://github.com/ogra1/nanopi-neo-gadget/blob/master/snapcraft.yaml

this individually downloads a cross compiler (the tree needs a newer version than the archive has, else you could just use gcc-arm-linux-gnueabihf) and does a cross build if it detectes that it is on x86 instead of armhf …

4

Is there any update on the cross compilation task. If I am for example planning to use Ubuntu Core on a ARM64 device how would my development workflow look like?

I have the following ideas in mind, maybe @ogra has some comments on them

  • Build your own sysroot based on debian dev packages to use the Linaro Cross toolchain. -> IMHO hard to maintain
  • Run a full Ubuntu Server instance in an ARM64 Qemu -> Probably slow as hell
  • Run a Ubuntu Server image on the target and perform the build there. -> Maybe you are running out of memory when building large applications and slow as hell
  • Run a LXD container on the target which hosts the CI/CD runner for automated builds and manual builds -> one additional abstraction.
  • Use the Launchpad build service. -> I do not want send my IP code to the cloud

The solution which seems to be the most interesting one:

  • Running a ARM64 buildserver in the company IT
5

what i do here at home is to run an ubuntu core image on top of a 4GB RPi4, having the writable partition (i.e. the whole OS) on an USB3.1 SSD … on top of this i run lxd containers… inside them i use snapcraft --destructive-mode to build my snaps …
this is about as fast as using build.snapcraft.io or launchpad.net directly.

i do have a few snaps that i cross compile but these are either kernels, bootloaders or minimal apps with very few (or no) dependencies … typically stuff that is designed to be cross-built from the ground up or easy enough to just call gcc -o foo foo.c