I am planning to use Posix message queues with snap for inter-snap communication.
But I observed that mq_open and other posix message queue syscalls are not allowed in seccomp filter.
Below lines in /var/lib/snapd/seccomp/bpf/snap.$SNAP_NAME.src indicates the same.
# LP: #1448184 - these aren’t currently mediated by AppArmor. Deny for now
I have referred to “https://snapcraft.io/docs/debug-snaps” and updated below files and re-generated bin file.
- Uncommented mq_open and other required syscalls in “/var/lib/snapd/seccomp/bpf/snap.$SNAP_NAME.src” to avoid seccomp violation.
- Added queue name with write/read permission in /var/lib/snapd/apparmor/profiles/snap.. to avoid apparmor violation.
With this modification, I am able to use posix message queue with snap. But these changes are not persistent and need to be done on each re-install.
Please clarify below queries.
a) What is the best way to make these changes persistent? Whether to create custom interface code and add it? or any other option? Please suggest.
b) POSIX message queues are not allowed by default where as Sys V message queue is allowed in seccomp filter policy. Any specific reason for this?
Thanks and Regards,
you’d actually create a new interface that explicitly allows using these message queues, then your app could just plug it …
Thanks for your reply.
Shall I refer to “How to create custom interface” for interface creation?
yeah, also take a look at former PRs in https://github.com/snapcore/snapd that add new interfaces …
Thanks for your suggestion.
I was able to create custom interface for message queue and currently testing the new interface by running modified snapd locally on my x86 system (Thanks to https://github.com/zyga/devtools/).
Next I need to integrate and verify the updated snapd on my arm64 platform. what are the possible options?
a) Whether I need to specify my forked snapd github path in any model file while creating Ubuntu core image snap?
b) Whether I need to cross compile, generate snapd.arm64 binary and pass as part of custom model assertion JSON file?
take a look that the --snap option of ubuntu-image , build your snapd snap for arm64 and pointing to the local .snap file should just work …
Thanks for your suggestion. I will try and come back if any further issues.
The current approach for allowing message queue needs modification in snapd (i.e. use of custom snapd).
Is there any other method available to allow posix message queue without modifying core snapd ?
Cross compiled snapd for arm64 (snapd.arm64) and able to build ubuntu core image with custom snapd with below command.
sudo ubuntu-image snap -d --image-size 4G --snap gadget.snap --snap kernel_5.4.85_arm64.snap --snap snapd.snap --output-dir image_out signed.model
Renamed snapd.arm64 to snapd.snap (as passing the file without rename introducing build errors)
Below is the snippet of model file used for including snapd.
Able to build and program the updated ubuntu core image into arm64 platform without any errors.
Below log is from device after programming ubuntu core image
core20 20210702 1084 latest/stable canonical✓ base
snapd 2.51.3 12707 latest/stable canonical✓ snapd
gadget 20-1 x1 - - gadget
kernel 5.4.85 x1 - - kernel
It looks like ubuntu core still fetching and using default snapd.
Please correct me if I missed any configuration.
well, running ubuntu-image with the debug option should actually show you if it downloads snapd or uses it from the local file (and it should also tell you that snapd will not be refreshed) … perhaps you can not override snapd itself from a local file, i have no experience with that since i rarely have a need to do such a thing …
BTW how exactly do you cross build snapd for arm64 ? AFAIK there is no support for such cross builds …
With debug option enabled, observing below prints during ubuntu core image build.
DEBUG:ubuntu-image:-> [ 0] make_temporary_directories
DEBUG:ubuntu-image:-> [ 1] prepare_gadget_tree
DEBUG:ubuntu-image:-> [ 2] prepare_image
WARNING: “kernel”, “gadget”, “app2-posix”, “app1-posix” installed from local snaps disconnected from a store cannot be refreshed subsequently!
Copying “kernel_5.4.85_arm64.snap” (kernel)
Copying “gdget_arm64.snap” (gadgeti)
Copying “app2-posix_1.0_arm64.snap” (app2-posix)
Copying “app1-posix_1.0_arm64.snap” (app1-posix)
“Fetching snapd” - indicates download of snapd.
Also Log indicate the copying of other local snaps (Eg: app1-posix, app2-posix) except snapd.
What is preferred way to use the custom snapd on Ubuntu core? Please suggest.
Cross compiled to arm64 using https://github.com/zyga/devtools/ . It will generate snapd.arm64 binary.
Created my own snapcraft.yaml for putting snapd binary into snap. Used snapcraft app type as snapd. Added script to copy snapd utility to /usr/lib/snapd.
You need to build the snapd snap, that link will only build snapd the binary executable. You can just use snapcraft to build the snapd snap for arm64.
Yes. After this, I have created my own snapcraft.yaml for inserting binary executable into snap. But this is not sufficient as snapd snap need to include other service files also.
I looks like current snapcraft.yaml of snapd do not have cross compilation support. Need some modification in yaml file to build for arm64.
What is the normally followed method to validate the changes done in snapd on platform?
Well what I usually do for development is to download the stable or edge channel snapd snap for the specified architecture and just replace the specific snapd binary with the one to be tested inside the snap and re-pack the snap. You can cross-compile the snapd binary since it is written in Go, though I think these days snapd the binary uses cgo so you also need a cross-compiling C toolchain available. Looking through my bash history this is the command I most recently used to build snapd for arm64:
GO111MODULE=off CGO_ENABLED=1 GOARCH=arm64 CC=aarch64-linux-gnu-gcc GOOS=linux go build -o snapd-arm64 ./cmd/snapd
Now, ideally you would run that inside a xenial 16.04 container or VM since it ends up linking against glibc and you want to be sure that it doesn’t depend on too new of a libc otherwise it will fail to start.
Hi, can you share your changes in these two files? i did in your way, but still got failure. Thanks.
My log output:
= Seccomp =
Time: Aug 26 08:17:44
Log: auid=4294967295 uid=0 gid=0 ses=4294967295 pid=776 comm=“myexample” exe="/snap/mydemo/x1/bin/myexample" sig=0 arch=40000028 277(mq_timedreceive) compat=0 ip=0xb6de9036 code=0x50000
Hi @ijohnson ,
Your suggested method works fine and I was able to validate custom snapd on arm64 platform.
What is the option if we plan to use custom snapd for production use case?
Well I hope you are not using a custom snapd for production? Can you share what changes you need to make that lead you to want to use a custom snapd? Perhaps we can upstream some of those changes so you can just use snapd from the store like everyone else and get all of our security updates automatically
Regrading getting this into an upstream interface, I pinged some relevant folks to see if any movement upstream in AppArmor happened to enable these message queues
Thanks for your support.
I will be interested in understanding the blockers which restricts the use of posix message queue within the snap.
Unfortunately, posix message queues are still not supported to be mediated in AppArmor, see bugs.launchpad.net/bugs/1448184 for details. You can mark yourself as affected by that bug, but until that bug is fixed and we have proper support in AppArmor for message queues like this, we cannot enable this support safely in snapd.