Unfortunately no. I will isolate the problem in another snap in a few days and i will give you that snap.
Same exact issue here. fs.copyFileSync fails in Snaps and works everywhere else - and all other Node fs methods we use work fine in Snaps even though copyFileSync fails.
We’re using Electron 17.x - which is based on Node 16.13.0 - so a year after this was initially reported, it remains an issue.
I suspect this is an issue affecting more projects than are actually reporting it - as we had no idea this was failing in Snaps until I spotted the issue in our error logs.
@mborzecki you can try our app here: https://snapcraft.io/recollectr
We expect this directory to contain backup files like: ~/snap/recollectr/current/.config/Recollectr/storage/backup/*.json - among other files, but instead it contains only the folder structure. Every filesystem operation works fine, but not copying.
Thanks for reporting. This is curious indeed. The seccomp profile has:
# the file descriptors used here will already be mediated by apparmor
# the 6th argument is flags, which currently is always 0
copy_file_range - - - - - 0
but in strace I can see:
pid 2073165] copy_file_range(30, [0], 39, NULL, 10836, 0) = -1 EPERM (Operation not permitted)
and there’s a log in dmesg:
auid=1000 uid=1000 gid=1000 ses=1
subj==snap.recollectr.recollectr (enforce) pid=2073165
comm="recollectr" exe="/snap/recollectr/33/recollectr"
sig=0 arch=c000003e syscall=326 compat=0 ip=0x7f9195364539 code=0x50000
syscall 326 on x86-64 is copy_file_range.
I first tried a rule like so:
copy_file_range - - - - - -
which worked, the call did not fail. Then, since the program always seems to be passing NULL as the 3rd argument, I tweaked the rule to:
copy_file_range - - - 0 - -
and it did not fail, strace log:
[pid 2088010] copy_file_range(30, [0], 39, NULL, 10836, 0) = 10836
Disassembly of the tweaked rule shows this:
0276: 0x15 0x00 0x03 0x00000146 jeq 326 true:0277 false:0280
0277: 0x20 0x00 0x00 0x0000002c ld $data[44]
0278: 0x15 0x00 0x32 0x00000000 jeq 0 true:0279 false:0329
0279: 0x05 0x00 0x00 0x00000197 jmp 0687
...
0329: 0x06 0x00 0x00 0x00050001 ret ERRNO(1)
...
0687: 0x20 0x00 0x00 0x00000028 ld $data[40]
0688: 0x15 0x6b 0x6a 0x00000000 jeq 0 true:0796 false:0795
...
0795: 0x06 0x00 0x00 0x00050001 ret ERRNO(1)
0796: 0x06 0x00 0x00 0x7fff0000 ret ALLOW
This is classic BPF, so 4 byte registers, arguments are 8 byte. It checks the first half of the arIt’s clearly not hitting the false branch in 278, then hits 687, checks the second half and reaches 796. The offsets seem to be correct, it’s loading halves of args[3] from seccomp_data structure as defined in https://elixir.bootlin.com/linux/v5.17.5/source/include/uapi/linux/seccomp.h#L60 (arg offsets would be 16, 24, 32, 40, 48, 56 AFAICT on x86-64)
Disassembling the bpf profile of the original rule (copy_file_range - - - - - 0):
$ scmp_bpf_disasm < /var/lib/snapd/seccomp/bpf/snap.recollectr.recollectr.bin
0276: 0x15 0x00 0x03 0x00000146 jeq 326 true:0277 false:0280
0277: 0x20 0x00 0x00 0x0000003c ld $data[60]
0278: 0x15 0x00 0x32 0x00000000 jeq 0 true:0279 false:0329
0279: 0x05 0x00 0x00 0x00000197 jmp 0687
...
0329: 0x06 0x00 0x00 0x00050001 ret ERRNO(1)
...
0687: 0x20 0x00 0x00 0x00000038 ld $data[56]
0688: 0x15 0x6b 0x6a 0x00000000 jeq 0 true:0796 false:0795
and yet somehow it does not match. This definitely needs more investigation. Maybe @alexmurray has some ideas about things I could try next.
For reference, the docker-support interface also allows copy_file_range, but without any arguments, which makes me curious as to why?. Perhaps a rule slimiar to what is in the base template did not work?
Edit:
With the problematic profile, the bpf simulator from libseccomp yields correct results:
$ scmp_bpf_sim -f /var/lib/snapd/seccomp/bpf/snap.recollectr.recollectr.bin \
-s 326 -0 29 -1 123 -2 39 -3 0 -4 10836 -5 0
ALLOW
$ ./scmp_bpf_sim -f /var/lib/snapd/seccomp/bpf/snap.recollectr.recollectr.bin \
-s 326 -0 29 -1 123 -2 39 -3 0 -4 10836 -5 1
ERRNO(1)
1 Like
For what it is worth, I’ve encountered similar errors while trying to get some parts of Steam to work under strict snap confinement. In this case, it was part of a Python script that used ctypes to invoke copy_file_range via syscall().
I extracted the relevant code out into a short test program here: https://paste.ubuntu.com/p/9XK8sft3Dc/
The weird thing is that it works when run using core20’s Python, but fails when using the SteamLinuxRuntime_soldier version. Using snap run --strace, the first gives:
openat(AT_FDCWD, "/tmp/foo", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/tmp/bar", O_WRONLY|O_CREAT|O_CLOEXEC, 0777) = 4
copy_file_range(3, NULL, 4, NULL, 1000, 0) = 6
and with the other:
openat(AT_FDCWD, "/tmp/foo", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/tmp/bar", O_WRONLY|O_CREAT|O_CLOEXEC, 0777) = 4
copy_file_range(3, NULL, 4, NULL, 1000, 0) = -1 EPERM (Operation not permitted)
The Steam runtime Python is 3.7.3, while core20’s is 3.8.10. Both are running against the core20 glibc.
2 Likes
Some further information: I can make the program work with the Steam Runtime Python if I modify the program to set the last argument to c_ulong instead of c_uint.
So my guess is that some garbage data next to the last argument is being passed through and tripping up the seccomp filter even though it’ll be ignored once the syscall is invoked.
After doing some more tests, I’m sure this is what’s happening. Here is a summary of what’s going on:
- The sixth argument to
copy_file_rangeisunsigned int flags, which is a 32-bit value even on 64-bit platforms. - The Linux calling conventions on x86-64 pass the syscall arguments in the registers %rdi, %rsi, %rdx, %r10, %r8 and %r9. These are 64-bit registers, so 32-bit arguments are passed in the low half.
- So on the kernel side, the
copy_file_rangeimplementation would read the flags argument from the low half of %r9. - The seccomp filter intercepts this invocation, and has no idea what the argument types are so just passes six 64-bit values to the BPF program.
- The BPF program snapd creates simply checks that
arg6 == 0, which passes if the high half of %r9 was cleared. But there is no guarantee that this is the case: it’s likely to hold whatever was in that register previously.
Tools like strace know to ignore the high 32-bits of the sixth argument when logging copy_file_range invocations, which is why the working and non-working invocations appeared identical to me.
libseccomp provides a seccomp.CompareMaskedEqual operator that lets us do a comparison against the low bits using a mask of 0xffffffff. And we’ve already got code in snap-seccomp using this for syscalls taking signed 32-bit values:
If I add copy_file_range to the syscallsWithNegArgsMaskHi32 map and recompile the seccomp filter for my snap, I stop seeing the EPERM errors. I realise flags is an unsigned value, but this part of the logic seems like it should apply for all 32-bit arguments.
Edit: I’ve created a PR to apply the simple fix here:
I’ve also filed an upstream libseccomp bug about improving handling of 32-bit arguments here:
While the CompareMaskedEqual code path works, it generates longer BPF than needed for the check.
4 Likes