Manual Review Request for box64-with-gl4es

box64 is a userspace x86_64 emulator for arm64… with a twist. Simply speaking, when it doesn’t emulate x86_64 instructions it attempts to load native versions of libraries the binary requires, enabling higher performance and compatibility.

It is best served as a binfmt.d handler, directly integrating the x86_64 execution support into the OS through a kernel interface. Though this kernel interface isn’t in use directly within the snap yet (no setup procedure right now) it is on the roadmap.

box64 needs access to various system files usually inaccessible to confined apps since the intended usecase is to run applications downloaded from the web, and those apps might require any possible combination of permissions unavailable to strict confinement, including access to either system-wide multiarch x86_64 libraries or some located in an arbitrary sysroot path. As such, I would like to request an exception for classic confinement for box64-with-gl4es.

From what I can see, as per the Process for reviewing classic confinement snaps, box64-with-gles does not appear to fit within one of the supported categories for classic confinement. Also note, that there are many other similar emulator snaps in the snap store (dosbox, scummvm, retroarch) none of which require classic confinement.

As such I do not feel this is appropriate for this snap.

I would like to pick discussions about this right back up, since I now intend to daily-drive my M1 MacBook with Ubuntu Asahi more and more.

The rationale for making it a classically confined app: box64 is not an emulator per se ala dosbox or scummvm, the term ‘interpreter’ is a technical one here. box64 as an interpreter allows to run arbitrary x86_64 binaries on arm64 hardware, much like Rosetta 2 from Apple on macOS. And much like Rosetta 2 it needs to be registered in a classically-confined app for it to allow executing things like OpenArena from the upstream .zip file.

The idea is to register binfmt misc handling using a setup script as part of the Snap, here is an example of how to register the classically confined Snap against x86_64 binaries on arm64:

:x86_64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00:\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/snap/bin/box64-with-gl4es.box64:

This results in x86_64 Linux binaries to be executable on arm64 requiring just the executable bit set for the binary file using chmod +x.

EDIT: Further context

The additional benefit is to run the actual host OSes library set against that interpreter because it hooks AMD64 function calls to arm64 wrappers, hence tricking the system into calling arm64 code. That works today and possibly allows to run on multiple distros supporting classic Snaps. It doesn’t necessarily mean putting “AMD64 on arm64 Linux” into snapd as it is but especially with some apps wanting to ship from “The Internet” it works very well (like the .AppImage’s and regular tarball binary releases).

I want to revive this topic because I want to bring box64 to Ubuntu users, now that I have a target machine to test this on again as my daily driver.

The current box64-with-gles status on GitHub allows for setting up box64 as a system-wide interpreter for Intel binary applications (like tar releases of upstream OpenArena or running Google kernel build toolchains which are Intel-only) capable of running on a Thinkpad x13s and allows for “Rosetta-like” functionality.

For this to work it needs some configuration dropped into the binfmt.d directory of the system. The result looks like this:

This allows for running the Intel-only kernel build scripts for compiling Ubuntu Touch-compatible kernels on ARM, transparently. See any difference to running native GCC? No? Then it works as intended. :slight_smile:

Please consider this usecase as part of classic confinement.

If you just want to drop something into the binfmt.d dir, the system-files interface might be sufficient, have you tried it ?

Dropping the configuration in /etc/binfmt.d consequently requires restarting systemd-binfmt, which cannot be done from a strictly confined snap.

interesting, i wonder why it doesnt use something like inotify to detect that stuff on its own and restart itself as needed …

It’s implemented as a oneshot, no idea.

oneshot should be fine, i was thinking along the lines to add PathChanged= to the unit in the deb so it triggers on changes to the dir … but apparenly PathChanged= can not be used in .service files, only in .path files …

https://manual.siduction.org/systemd-path_en.html#path-unit-options

Well if it means shipping an additional .path file which restarts systemd-binfmt, sure. But I can see upstream wanting an argument here, might be reasonable proposing that with an invitation to talk about a solution. Or maybe a patch to systemd which allows globbing PathChanged for a snap-specific file prefix. But that all sounds like mit-to-long-term solutions.

you can indeed not ship .path files in snaps (i mean, yes, you can but they wouldnt be in the right place anyway), what i meant was to file an upstream bug/issue asking for either systemd-binfmt to learn using inotify in the binary or to have said .path file.

from the snap side there is indeed not much you could do … but i also do not think having to restart binfmt will be enough to get classic granted …

there might be a way through adding a new snapd interface or some such, that does allow writing to /etc/binfmt.d and that automatically triggers the restart as needed …

I am not sure whether running an Intel binary from within a strict sandboxes’ context makes sense though when the rest of the machinery at hand (AOSP or Halium build scripts) has expectations like that. Ideally it would transparently work like any other regular ARM binary on the ARM box.

there are plenty of strictly confined VM snaps in the store that run fine in a “strict sandbox” context.

i think adding some “binfmt-support” interface that allows dropping files into /etc/binfmt.d at snap install time and some code in the interface that does the systemd-binfmt restart call after dropping the file in would be the best way forward currently …

Here is my case for why it doesn’t have to be strict with the following personal use case:

I want to install a classic exe game inside of my Wine prefix that I have on the host. The use of strict confinement would reduce access to the various hardware and capabilities as all of the sudden a regular Wine app would be forcefully put into the strict context. The ultimate goal is to have Wine be packaged outside of the snap and inside of the host, and enhancing the ability to run Intel binaries through a Snap regardless.

If for that the necessary interfaces aren’t there then classic would be the only way forward, for now.

For wine you should be able to use the personal-files interface …

plugs:
    dot-wine:
      interface: personal-files
      write:
         - $HOME/.wine

Okay, after some consideration & experimentation I was able to package it up as strictly confined, at last. It required packaging as many supported libraries as possible, but I’ve made use of the gnome extension and mesa content snaps to check all the compatibility checkmarks.

I will go ahead and release it as is with the setup script merely printing the instructions on how to enable it system-wide manually. Best case I could get granted permissions for a plug in the configure hook allowing the Snap to do what is necessary to set up the environment but for now this suffices.

Thanks for the healthy discussion. If there are ways to help improve the box64 Snap setup experience we can open a new issue for it.

EDIT: Dare I say it, this has been a carefully crafted snap.

Well dang it, will take some time since it might be stuck from previous release attempts going classic and getting flagged?

If you’ve releases stuck in the moderation queue you can manually remove them from the queue at https://dashboard.snapcraft.io - you should then be able to upload another revision (but naturally the revision that’s flagged for moderation won’t be promotable)

Thanks, I was able to clear the queue and push a revision without requiring any privileged interfaces. It’s available here: Install box64-with-gl4es on Linux | Snap Store

2 Likes

@beidl it seems box64-with-gl4es has been successfully published under strict confinement without further privileges access required. That’s great!

I am now removing this request from our review queue, but please feel free to write here again if you have any further question.

Thanks!