I am planning to use Posix message queues with snap for inter-snap communication.
But I observed that mq_open and other posix message queue syscalls are not allowed in seccomp filter.
Below lines in /var/lib/snapd/seccomp/bpf/snap.$SNAP_NAME.src indicates the same. # LP: #1448184 - these aren’t currently mediated by AppArmor. Deny for now #mq_getsetattr #mq_notify #mq_open #mq_timedreceive #mq_timedreceive_time64 #mq_timedsend #mq_timedsend_time64 #mq_unlink
Uncommented mq_open and other required syscalls in “/var/lib/snapd/seccomp/bpf/snap.$SNAP_NAME.src” to avoid seccomp violation.
Added queue name with write/read permission in /var/lib/snapd/apparmor/profiles/snap.. to avoid apparmor violation.
With this modification, I am able to use posix message queue with snap. But these changes are not persistent and need to be done on each re-install.
Please clarify below queries.
a) What is the best way to make these changes persistent? Whether to create custom interface code and add it? or any other option? Please suggest.
b) POSIX message queues are not allowed by default where as Sys V message queue is allowed in seccomp filter policy. Any specific reason for this?
I was able to create custom interface for message queue and currently testing the new interface by running modified snapd locally on my x86 system (Thanks to https://github.com/zyga/devtools/).
Next I need to integrate and verify the updated snapd on my arm64 platform. what are the possible options?
a) Whether I need to specify my forked snapd github path in any model file while creating Ubuntu core image snap?
b) Whether I need to cross compile, generate snapd.arm64 binary and pass as part of custom model assertion JSON file?
Thanks for your suggestion. I will try and come back if any further issues.
The current approach for allowing message queue needs modification in snapd (i.e. use of custom snapd).
Is there any other method available to allow posix message queue without modifying core snapd ?
well, running ubuntu-image with the debug option should actually show you if it downloads snapd or uses it from the local file (and it should also tell you that snapd will not be refreshed) … perhaps you can not override snapd itself from a local file, i have no experience with that since i rarely have a need to do such a thing …
BTW how exactly do you cross build snapd for arm64 ? AFAIK there is no support for such cross builds …
With debug option enabled, observing below prints during ubuntu core image build.
DEBUG:ubuntu-image:-> [ 0] make_temporary_directories DEBUG:ubuntu-image:-> [ 1] prepare_gadget_tree DEBUG:ubuntu-image:-> [ 2] prepare_image Fetching snapd Fetching core20 WARNING: “kernel”, “gadget”, “app2-posix”, “app1-posix” installed from local snaps disconnected from a store cannot be refreshed subsequently! Copying “kernel_5.4.85_arm64.snap” (kernel) Copying “gdget_arm64.snap” (gadgeti) Copying “app2-posix_1.0_arm64.snap” (app2-posix) Copying “app1-posix_1.0_arm64.snap” (app1-posix)
“Fetching snapd” - indicates download of snapd.
Also Log indicate the copying of other local snaps (Eg: app1-posix, app2-posix) except snapd.
What is preferred way to use the custom snapd on Ubuntu core? Please suggest.
Cross compiled to arm64 using https://github.com/zyga/devtools/ . It will generate snapd.arm64 binary.
Created my own snapcraft.yaml for putting snapd binary into snap. Used snapcraft app type as snapd. Added script to copy snapd utility to /usr/lib/snapd.
Yes. After this, I have created my own snapcraft.yaml for inserting binary executable into snap. But this is not sufficient as snapd snap need to include other service files also.
I looks like current snapcraft.yaml of snapd do not have cross compilation support. Need some modification in yaml file to build for arm64.
What is the normally followed method to validate the changes done in snapd on platform?
Well what I usually do for development is to download the stable or edge channel snapd snap for the specified architecture and just replace the specific snapd binary with the one to be tested inside the snap and re-pack the snap. You can cross-compile the snapd binary since it is written in Go, though I think these days snapd the binary uses cgo so you also need a cross-compiling C toolchain available. Looking through my bash history this is the command I most recently used to build snapd for arm64:
GO111MODULE=off CGO_ENABLED=1 GOARCH=arm64 CC=aarch64-linux-gnu-gcc GOOS=linux go build -o snapd-arm64 ./cmd/snapd
Now, ideally you would run that inside a xenial 16.04 container or VM since it ends up linking against glibc and you want to be sure that it doesn’t depend on too new of a libc otherwise it will fail to start.
Your suggested method works fine and I was able to validate custom snapd on arm64 platform.
What is the option if we plan to use custom snapd for production use case?
Well I hope you are not using a custom snapd for production? Can you share what changes you need to make that lead you to want to use a custom snapd? Perhaps we can upstream some of those changes so you can just use snapd from the store like everyone else and get all of our security updates automatically
Regrading getting this into an upstream interface, I pinged some relevant folks to see if any movement upstream in AppArmor happened to enable these message queues
Unfortunately, posix message queues are still not supported to be mediated in AppArmor, see bugs.launchpad.net/bugs/1448184 for details. You can mark yourself as affected by that bug, but until that bug is fixed and we have proper support in AppArmor for message queues like this, we cannot enable this support safely in snapd.
…