Kernel Snap fails to build

wililupy · January 3, 2018, 7:05pm

I’m trying to build a kernel snap and I get the following error:

Generating driver initrd for kernel release: 4.4.105-rt120
Parallel unsquashfs: Using 4 processors
2 inodes (2 blocks) to write

[===================================================================|] 2/2 100%

created 0 files
created 4 directories
created 2 symlinks
created 0 devices
created 0 fifos
Traceback (most recent call last):
  File "/usr/bin/snapcraft", line 9, in 
    load_entry_point('snapcraft==2.35', 'console_scripts', 'snapcraft')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 542, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2569, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2229, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2235, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/snapcraft/cli/__main__.py", line 19, in 
    run(prog_name='snapcraft')
  File "/usr/lib/python3/dist-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1037, in invoke
    return Command.invoke(self, ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3/dist-packages/click/core.py", line 86, in augment_usage_errors
    yield
  File "/usr/lib/python3/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/lib/python3/dist-packages/snapcraft/cli/__init__.py", line 124, in run
    ctx.forward(lifecyclecli.commands['snap'])
  File "/usr/lib/python3/dist-packages/click/core.py", line 552, in forward
    return self.invoke(cmd, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3/dist-packages/click/core.py", line 86, in augment_usage_errors
    yield
  File "/usr/lib/python3/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/snapcraft/cli/lifecycle.py", line 140, in snap
    project_options, directory=directory, output=output)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle/_packer.py", line 45, in snap
    execute('prime', project_options)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle/_runner.py", line 80, in execute
    _Executor(config, project_options).run(step, part_names)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle/_runner.py", line 175, in run
    self._run_step(step, part, part_names)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle/_runner.py", line 212, in _run_step
    getattr(part, step)()
  File "/usr/lib/python3/dist-packages/snapcraft/internal/pluginhandler/__init__.py", line 330, in build
    self.plugin.build()
  File "/usr/lib/python3/dist-packages/snapcraft/plugins/kbuild.py", line 258, in build
    self.do_install()
  File "/usr/lib/python3/dist-packages/snapcraft/plugins/kernel.py", line 469, in do_install
    self._make_initrd()
  File "/usr/lib/python3/dist-packages/snapcraft/plugins/kernel.py", line 282, in _make_initrd
    initrd_unpacked_path = self._unpack_generic_initrd()
  File "/usr/lib/python3/dist-packages/snapcraft/plugins/kernel.py", line 255, in _unpack_generic_initrd
    tmp_initrd_path, os.strerror(mime_detector.errno())))
RuntimeError: Unable to determine mime type for '/tmp/tmphywibpjn/squashfs-root/usr/lib/ubuntu-core-generic-initrd/initrd.img-core': No such file or directory

Not quite sure where to go next? It has been a while since I have done this, so here is my snapcraft.yaml in my snap/ directory:

name: rt-kernel
version: '4.4.105' 
summary: Custom Kernel snap for Ubuntu-Core
description: Customer Kernel based on upstream 4.4.105 kernel and patched with RT patch from Kernel.org.
grade: stable
confinement: strict
type: kernel

parts:
  kernel:
    plugin: kernel
    source: linux-4.4.105/
    kconfigfile: RT-Config
    kernel-initrd-modules: 
       - squashfs
       - ahci

Please let me know if you need anything else.

Thanks,
Luke

sergiusens · January 3, 2018, 8:39pm

The core snap changed due to reasons unbeknownst to me, @ppisati has created a PR to solve that PR: #1839

ogra · January 4, 2018, 12:39pm

Seems this was for better bsdiff generation in the store … (also seems that nobody checked with snapcraft before landing it )

github.com/canonical/core

25-create-generic-initrd.chroot: use symlink instead of copy

canonical:master ← mvo5:one-initrd-ought-to-be-enough

opened 02:54PM - 10 Nov 17 UTC

mvo5

+4 -1

There was a discussion about the core snap files recently. One of the observati…ons was that we currently ship two initrd.img images. This is not a problem as such as squashfs compresses the two identical files away. However it is a problem when a delta is generated via the `bsdiff` mechanism. This method will look into the squashfs data itself and because initrd is compressed it apparently has trouble generating a optional delta.

mvo · January 4, 2018, 4:43pm

I created https://github.com/snapcore/core/pull/67 to change the symlink ordering. However if this is fixed in snapcraft that is even better.

sergiusens · January 4, 2018, 6:08pm

only one solution must survive for this not to break again

sergiusens · January 4, 2018, 10:00pm

PR: #1839 is in

wililupy · January 16, 2018, 6:25pm

So I upgraded my snapcraft to the edge channel (snapcraft, version 2.38+git28.4c7edab) and I still get the same error:

  DEPMOD  4.4.105-rt120
Generating driver initrd for kernel release: 4.4.105-rt120
Parallel unsquashfs: Using 4 processors
2 inodes (33 blocks) to write

[=================================================================|] 33/33 100%

created 1 files
created 6 directories
created 1 symlinks
created 0 devices
created 0 fifos
25725 blocks
Traceback (most recent call last):
  File "/snap/snapcraft/1033/bin/snapcraft", line 11, in 
    load_entry_point('snapcraft==2.38', 'console_scripts', 'snapcraft')()
  File "/snap/snapcraft/1033/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 565, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/snap/snapcraft/1033/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
    return ep.load()
  File "/snap/snapcraft/1033/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/snap/snapcraft/1033/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/cli/__main__.py", line 19, in 
    run(prog_name='snapcraft')
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/click/core.py", line 1043, in invoke
    return Command.invoke(self, ctx)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/cli/_runner.py", line 72, in run
    ctx.forward(lifecyclecli.commands['snap'])
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/click/core.py", line 553, in forward
    return self.invoke(cmd, **kwargs)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/cli/lifecycle.py", line 139, in snap
    project_options, directory=directory, output=output)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_packer.py", line 45, in snap
    execute('prime', project_options)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 79, in execute
    _Executor(config, project_options).run(step, part_names)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 184, in run
    self._run_step(step, part, part_names)
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/internal/lifecycle/_runner.py", line 221, in _run_step
    getattr(part, step)()
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 362, in build
    self.plugin.build()
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/plugins/kbuild.py", line 258, in build
    self.do_install()
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/plugins/kernel.py", line 456, in do_install
    self._make_initrd()
  File "/snap/snapcraft/1033/lib/python3.6/site-packages/snapcraft/plugins/kernel.py", line 284, in _make_initrd
    os.link(src, dst)
FileNotFoundError: [Errno 2] No such file or directory: 'squashfs' -> '/home/wililupy/rt-kernel/parts/kernel/build/initrd-staging/../../../squashfs'

Any ideas?

Thanks,
Luke

ogra · January 16, 2018, 8:49pm

are you sure you use the snap (works fine here since the change landed) and not a parallel installed deb ?

wililupy · January 16, 2018, 10:02pm

@ogra

Yes. I removed the snapcraft deb using apt remove --purge snapcraft && apt
autoremove --purge and then I installed the snapcraft snap using sudo snap
install snapcraft --edge --classic and it still fails to build. The kernel
itself builds fine and runs, but for some reason I can’t build it in
snapcraft.

I may try a clean system/container and give it another shot.

Thanks,
Luke

wililupy · January 16, 2018, 11:28pm

I think I might have figured it out. It is looking for the squashfs initrd module, yet I’m not building squashfs as a module. I removed squashfs from the kernel-initrd-modules in my snapcraft.yaml.

I’m rebuilding now to see if it works. I’ll let everyone know.

Thanks,
Luke

wililupy · January 17, 2018, 12:07am

That was it. I was able to successfully build the kernel snap.

Thanks everyone!