I’m working on creating a snap for Autopsy, which runs on the Netbeans Rich Client Platform, which in turn, uses java 17 along with a fair number of native dependencies through jni and calling external processes using something like ProcessBuilder. Given the system access that Autopsy requires to do what it does, I’m using classic confinement. I’m using snapcraft version 7.4.3 to build the snap package, and using a base of core22 with this snapcraft.yaml.
I’m trying to ensure that the snap uses shared object libraries within the snap and hitting a lot of issues. Based on this article, I’m using
build-attributes:
- enable-patchelf
in my parts because my snapcraft version is > 7.3, and I have played around with the LD_LIBRARY_PATH as well. Unfortunately, I’m hitting lots of issues with what appears to be finding the right version of glib with a message like this:
/snap/autopsy/x1/usr/lib/jvm/java-17-openjdk-amd64/bin/java: symbol lookup error: /lib/x86_64-linux-gnu/libpthread.so.0: undefined symbol: __libc_pthread_init, version GLIBC_PRIVATE
I don’t know if this helps, but I ran LD_DEBUG but it looks like it is using the wrong version libpthread initially:
and after playing with the LD_LIBRARY_PATH to add the core22x86_64-linux-gnu folder,
I get issues resolving the right version of ld-linux-x86-64:
5213: file=libc.so.6 [0]; needed by autopsy [0]
5213: find library=libc.so.6 [0]; searching
5213: search path=/snap/core22/current/lib64:/snap/autopsy/current/usr/local/lib:/snap/core22/current/usr/lib/x86_64-linux-gnu:tls/x86_64/x86_64:tls/x86_64:tls/x86_64:tls:x86_64/x86_64:x86_64:x86_64: (LD_LIBRARY_PATH)
5213: trying file=/snap/core22/current/lib64/libc.so.6
5213: trying file=/snap/autopsy/current/usr/local/lib/libc.so.6
5213: trying file=/snap/core22/current/usr/lib/x86_64-linux-gnu/libc.so.6
5213:
5213: file=libc.so.6 [0]; generating link map
5213: dynamic: 0x00007f1df78c3bc0 base: 0x00007f1df76ab000 size: 0x0000000000227e50
5213: entry: 0x00007f1df76d4f50 phdr: 0x00007f1df76ab040 phnum: 14
5213:
5213: checking for version `GLIBC_2.3.2' in file /snap/core22/current/usr/lib/x86_64-linux-gnu/libpthread.so.0 [0] required by file autopsy [0]
5213: checking for version `GLIBC_2.2.5' in file /snap/core22/current/usr/lib/x86_64-linux-gnu/libpthread.so.0 [0] required by file autopsy [0]
5213: checking for version `GLIBC_2.4' in file /snap/core22/current/usr/lib/x86_64-linux-gnu/libc.so.6 [0] required by file autopsy [0]
5213: checking for version `GLIBC_2.3.4' in file /snap/core22/current/usr/lib/x86_64-linux-gnu/libc.so.6 [0] required by file autopsy [0]
5213: checking for version `GLIBC_2.2.5' in file /snap/core22/current/usr/lib/x86_64-linux-gnu/libc.so.6 [0] required by file autopsy [0]
5213: checking for version `GLIBC_2.2.5' in file /snap/core22/current/usr/lib/x86_64-linux-gnu/libc.so.6 [0] required by file /snap/core22/current/usr/lib/x86_64-linux-gnu/libpthread.so.0 [0]
5213: checking for version `GLIBC_2.2.5' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /snap/core22/current/usr/lib/x86_64-linux-gnu/libc.so.6 [0]
5213: checking for version `GLIBC_2.3' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /snap/core22/current/usr/lib/x86_64-linux-gnu/libc.so.6 [0]
5213: checking for version `GLIBC_PRIVATE' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /snap/core22/current/usr/lib/x86_64-linux-gnu/libc.so.6 [0]
Is there something I’m missing in this process or a better way of approaching this?
Unfortunately I’ve never walked the classic side so I can’t be too helpful; but I think generally you want to avoid LD_LIBRARY_PATH for classic snaps because the env var will propogate to processes launched via the snap, most of which probably will fail since they’re unlikely to be ABI compatible with the core runtime the snap itself is using.
Unfortunately I can’t say much more than that, but I’d probably start by looking at /snap/autopsy/x1/usr/lib/jvm/java-17-openjdk-amd64/bin/java with the readelf command and ensuring rpath/runpath is correctly set on the binary; which should be what’s being used in lieu of LD_LIBRARY_PATH.
Thanks for your thoughts. I’ll look into the rpath/runpath a little more. So, you think this is an instance that enable-patchelf doesn’t catch or handle appropriately that needs to be handled more manually?
I’m thinking that perhaps you need to avoid automatic patching and just invoke patchelf at the right point in the build process.
I think that callingsnapcraft build is fine, though I think you are doing most of the work yourself given that you are using the nil plugin.
Thanks once again. I tried backing down to core20, and I didn’t hit quite these problems anymore (https://github.com/gdicristofaro/autopsy/blob/8425c-core20ElfFix/snap/snapcraft.yaml). I’m not entirely sure why that would be. The application launches, but it is hitting quite a few problems around gtk lib loading. I tried incorporating some gtk libraries in my stage-packages, and that got me quite a bit further in that the app is behaving much closer to usual, but still a few more bugs to work out. Is there a good way to identify whether a classic snap is behaving in an encapsulated manner? Is it just checking the ld output to see what libs it is loading or installing on multiple distros?
Which version of Ubuntu are you developing on? I wonder if you are seeing issues because, as a classic snap, the application is able to use some libraries that it would use if built as a non-snap application, but not everything it needs.
I’m building on Ubuntu 22.04 and testing on Ubuntu 22.04 as well as 20.04. I’m playing around with strict and devmode confinement now to see if that can cover my needs. I think I initially underestimated its capability, probably would be easier to get into the store if we end up going that route in the future, and from my initial attempts it has helped me sidestep a lot of these issues. I can start a new forum post if that would be better, but there are a few things I was wondering about regarding strict confinement access.
The application offers the capability of processing local disks which we process from /dev/disk or /dev/sda*. I think these require disk group access. This may just be my lack of knowledge or research, but is there a way to run a strictly confined app such that it has root or a certain group’s permissions? At any rate, is there a way to access those block devices?
As a part of the application, we run Solr which runs on localhost and then we check to see if it has binded to the port before proceeding. There are two issues I’ve identified thus far. One of which is that we are running solr with the jvm option of -XX:+UseLargePages, which from snappy-debug appears to cause a violation in accessing /sys/kernel/mm/hugepages. Is there any way around that? Also, the script that we have to check if Solr is up and running is using a command like lsof -t -PniTCP:$SOLR_PORT -sTCP:LISTEN, which is failing right now. As best I can tell, one of these is the issue:
= AppArmor =
Time: Jul 17 09:11:49
Log: apparmor="DENIED" operation="open" profile="snap.autopsy.autopsy" name="/proc/13690/fdinfo/3" pid=13690 comm="lsof" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000
File: /proc/13690/fdinfo/3 (read)
Suggestions:
* adjust program to not access '@{PROC}/@{pid}/fdinfo/3'
* adjust program to not access '@{PROC}/@{pid}/fdinfo/[0-9]*'
= AppArmor =
Time: Jul 17 09:11:49
Log: apparmor="DENIED" operation="open" profile="snap.autopsy.autopsy" name="/proc/locks" pid=13690 comm="lsof" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0
File: /proc/locks (read)
Suggestions:
* adjust program to not access '@{PROC}/locks'
* add 'system-observe' to 'plugs'
= AppArmor =
Time: Jul 17 09:11:49
Log: apparmor="DENIED" operation="ptrace" profile="snap.autopsy.autopsy" pid=13690 comm="lsof" requested_mask="read" denied_mask="read" peer="unconfined"
Ptrace: peer=unconfined (read)
Suggestions:
* add 'system-observe' to 'plugs'
* do nothing if program otherwise works properly
Sorry for the late reply. I got bogged down with other work until now. Thank you very much for all of your help, and for the suggestion with hugepages-control. I’m still hitting issues with hugepages access. Looking through the open jdk code here, it looks like the /sys/kernel/mm/hugepages directory gets scanned for anything that looks like hugepages-[0-9]*kB, but that seems to be denied:
= AppArmor =
Time: Jul 31 07:52:53
Log: apparmor="DENIED" operation="open" profile="snap.autopsy.autopsy" name="/sys/kernel/mm/hugepages/" pid=70889 comm="java" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0
File: /sys/kernel/mm/hugepages/ (read)
Suggestion:
* adjust program to not access '/sys/kernel/mm/hugepages/'
I also tried adding a ‘system-files’ rule like this:
I may not be entirely tracking what you’re saying, but I think writing requires root permissions and any can read. This is what I get running on Xubuntu 22.04:
ls -l /sys/kernel/mm | grep hugepages
drwxr-xr-x 3 root root 0 Jul 31 10:54 hugepages
ls -l /sys/kernel/mm/hugepages
total 0
drwxr-xr-x 2 root root 0 Jul 31 10:55 hugepages-2048kB
Well, having another chance to prove my blindness here, but it doesn’t look like your interfaces are actually connected … try to run snap connect.... for them again …
(The third column usually shows the target, your grep output only shows " - " there)
Thank you, yes, I think I was missing the snap connect ... bit of this. I initially got the same apparmor error after running just snap connect autopsy:hugepages-control. I added another interface like this:
, ran snap connect ..., and that was able to get me the rest of the way there. I may still be missing something in regards to hugepages-control, so if there is a way I can better integrate that without the additional interface declaration that would be great too.
I did a bit of reading here, and I think I misunderstood that not all interfaces are connected during install. My initial goal for the snap is to be able to generate a snap package that users can fairly easily install on their devices, and hopefully, at some later date, get the package in the store. For the time being though, if I’m using snap try... for debugging and development or we want users to be able to easily install the package and connect all these interfaces, is there some sort of --connect-all flag or something like that so that all declared plugs including super-priveleged ones get connected?
If you’re distributing outside the store (or inside it too), you can mark the confinement as devmode; users will have to pass --devmode when installing the snap (in addition to --dangerous, if it’s outside the store), but this effectively disables most sandboxing (primarily the ones interfaces would be concerned with, it’s still not the same as classic, e.g, you’re still going to be in a mount namespace).
The downside of --devmode is there’s no automatic updates, even when distributed via the store.
Bear in mind, you can ask for the store to have an override that means those interfaces will automatically connect, subject to a review/approval process, which might help you out for proper strict confinement.
, sorry for the noise