I’m running into some errors while attempting to upgrade my snap to strict confinement for deployment to the snap store. Everything is working happily in dev mode. We’re using Ubuntu Server 16.04 and do not plan to move to Core any time soon.
My snap is comprised of a few straightforward ROS nodes but it depends on something called EtherCAT, which I build outside of the snap. I receive a few errors when the snap starts. Originally, I was receiving close to a dozen but have it down to four since connecting the mount-observe interface. The remaining errors are below.
Jan 22 13:27:15 bbi-rosbox-omega kernel: [ 512.995233] audit: type=1400 audit(1516645635.495:45): apparmor="DENIED" operation="connect" profile="snap.mysnap.ros" name="/run/uuidd/request" pid=3555 comm="python" requested_mask="wr" denied_mask="wr" fsuid=0 ouid=0
Jan 22 13:27:15 bbi-rosbox-omega kernel: [ 513.028424] audit: type=1400 audit(1516645635.527:46): apparmor="DENIED" operation="exec" profile="snap.mysnap.ros" name="/usr/bin/du" pid=3669 comm="python" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
Jan 22 13:27:16 bbi-rosbox-omega kernel: [ 513.534355] audit: type=1400 audit(1516645636.035:47): apparmor="DENIED" operation="capable" profile="snap.mysnap.ros" pid=3705 comm="proteus_control" capability=1 capname="dac_override"
Jan 22 13:27:16 bbi-rosbox-omega kernel: [ 513.534362] audit: type=1400 audit(1516645636.035:48): apparmor="DENIED" operation="capable" profile="snap.mysnap.ros" pid=3705 comm="proteus_control" capability=2 capname="dac_read_search"
Jan 22 13:27:16 bbi-rosbox-omega mysnap.ros[3555]: Failed to open /dev/EtherCAT0: Operation not permitted
Jan 22 13:27:16 bbi-rosbox-omega kernel: [ 513.629819] proteus_control[3705]: segfault at 0 ip 00007f2823103c86 sp 00007ffc79c476f0 error 4 in libethercat.so.1[7f2823101000+8000]
The only other
Strangely, the Failed to open /dev/EtherCAT0 message alternates between Operation not permitted and Permission denied.
The interfaces reference makes it clear that there are interfaces to access other devices in /dev, but I am unclear on how I’d grant access to something not already provided by an existing interface. This excellent tutorial provides a great reference for creating an interface and I could easily copy/paste/modify an existing one to point to my device, but I see nothing about redistributing and installing custom interfaces. Unclear about whether this is the right way to go.
As for the other errors, the python ones, I am not sure how to proceed. It’s likely that those commands are being executed by dependencies of our ROS packages.
I thank you in advance for any attention and help you can provide!
First of all, have you seen this debugging documentation? Specifically the bit about using snappy-debug, which is a super handy tool. It’s definitely something to have in your toolbox, although I’m not sure how helpful it will be with these specific denials.
I believe this is attempting to get a UUID from the uuid daemon running on the host (according to man uuidd, its socket is /run/uuidd/request). You might consider bundling this daemon in your snap; then you can ask it to put its socket in, say, /tmp using the -s option. You also need to tell the client to use it though-- is it something you wrote yourself?
Hey Kyle, thanks for checking these out. I’m going to start with reviewing snappy-debug. In the meantime…
RE: the UUID, that is unfortunately not coming from anything we wrote. I’m guessing it’s a one of our ROS dependencies, maybe Rosbridge?
RE: the two dac_ errors, my only guess is that it’d have something to do with EtherCAT. That particular package is the one that is responsible for communicating with the robot using that interface. Oddly enough, those errors do not appear consistently. More often than not, they do not appear at all.
Alright, let’s figure that out, I’ll do some research.
This may be a fallback, then, as a side effect of not being able to access the ethercat device. Let’s focus on getting access to that device, and we can revisit this one if necessary.
Ah, this is a red herring. roslaunch uses uuid.uuid1() to generate run IDs, which apparently (as judged by my own tests using uuid.uuid1()) attempts to use uuidd, realizes it can’t, and generates its own. I think you can safely ignore the /run/uuidd/request denial.
snappy-debug is screaming about one of the ethercat dependencies in my home folder. I just added the home plug and moved the files it needs out from a hidden folder. Rebuilding now and we’ll see what we get.
uuidd from the uuid-runtime package listens on this socket. It is not on Ubuntu Core and I agree with @kyrofa that shipping it in your snap or falling back as later found is probably the best way to solve this.
/usr/bin/du is an omission from the default template. I’ve added a todo for this and it will be fixed in the next batch of policy updates.
dac_override and dac_read_search you should be able to solve in your snap. These are coming from the fact that a root process is trying to access files and read directories that it wouldn’t normally have access to, except for the fact it is root. Often this is caused by the root process reading files that are owned by a non-root owner and group with too strict ‘other’ permissions.
/dev/EtherCAT0 is not something that is available in snapd currently. I don’t have experience with this, but it seems like you need both kernel support, udev rules to reveal the devices and userspace to access them. Our reference kernel snaps don’t currently support these devices. For snapd to support it, we’d need to add them to the kernel snaps and create/update an interface to add the udev rules and access to the devices. Can you provide documentation on ethercat?
Hey @jdstrand, thanks for weighing in! I’m not seeing those dac_ errors consistently, which makes me wonder if it might end up working normally if we can get EtherCAT working.
I can provide you with plenty of information about EtherCAT but to be honest, it’s so uncommon that it feels really weird to expect it to be part of snapd. Do I have any options for working around this without requiring work on your end? This snap will never be released publicly, if that gives us more options when it comes to bending rules about what is and isn’t permitted in a stable channel snap.
Can also report that snappy-debug is no longer complaining about anything other than /usr/bin/du, though my syslog does still report Failed to open /dev/EtherCAT0: Operation not permitted, as we would expect. If it matters at all, I have an existing udev rule that lets everything work when I’m in dev mode. Is there any way to use this in conjunction with other permission changes to allow permission from within the snap?
Would classic confinement be a better fit than strict in this case?
EDIT: It appears that this might fix my EtherCAT problems but the change of paths breaks ROS’s ability to start. Not sure which is easier to fix but I’m guessing it might be ROS. Eager to know what you guys think.
Only devmode. There isn’t a way to dynamically add interfaces to the system (this is intentional in the design). For quick hacks, you can update your profiles in /var/lib/snapd/apparmor/profiles/snap… and use sudo apparmor_parser -r /var/lib/snapd/apparmor/profiles/snap... to load them into the kernel, but snapd will periodically regenerate the profiles and overwrite your changes.
Adding an interface to snapd is not particularly difficult (as you alluded to before) and I think an ethercat interface seems fine (there are plenty of specialist interfaces). If you are considering implementing it, I suggest the name ethercat-control.
I definitely can’t have things periodically breaking on these systems. Adding the interface seems simple enough, but I haven’t been able to find anything about how to redistribute my new interface to dependent systems once I’ve built it. The tutorial I linked to earlier makes it look like I’d need all of my subscribers to depend on my fork of snapd, which seems odd. Is there more documentation about this somewhere?
If you add the interface to snapd, then it will be in a future core snap update and since devices auto-refresh, all devices will get the new interface.
You could use classic, but classic can’t be used on Ubuntu Core devices plus classic confinement requires a snap declaration in the store since it allows you to publish to the stable channel. Since you said you didn’t really want this distributed via the store, it sounded like devmode would be a better fit than classic.
(That said, an ethercat-control interface sounds like it would benefit you and all ethercat users-- thanks @kyrofa for the offer of help on the interface!)
