Hello.
I’m trying to migrate Automount for Ubuntu Core to core24 but the mount-control interface isn’t working.
System call co mount() and command mount return error 1: Permission denied, but snapctl mount works properly.
Is there a specific AppArmor denial you see? What kind of flags are you passing with the mount?
snapctl mount results in a systemd unit being created (transient or persistent, depending on what is requested), so the mechanism is quite different.
Also, please include the output of snap list and snap version.
No denial is shown.
the commands are:
mount -t vfat /dev/sdd /media/auto/usb -o rw,sync Error
snapctl mount -t vfat /dev/sdd /media/auto/usb -o rw,sync OK
$ snap list
Name Version Rev Tracking Publisher Notes
automount-core 0.1.1 11 latest/candidate edv -
console-conf 24.04.1+git45g5f9fae19+gd81a15d 41 24/stable canonical✓ -
core24 20240426 410 latest/stable canonical✓ base
pi 24-1 142 24/stable canonical✓ gadget
pi-kernel 6.8.0-1004.4 825 24/stable canonical✓ kernel
snapd 2.62 21467 latest/stable canonical✓ snapd
$ snap version
snap 2.62
snapd 2.62
series 16
kernel 6.8.0-1004-raspi
Can you include output from dmesg and the exact error message you observed?
I just tried this in a UC24 amd64 VM:
maciek-borzecki@localhost:~$ snap list
Name Version Rev Tracking Publisher Notes
automount-core 0.1.1 17 latest/candidate edv -
console-conf 24.04.1+git45g5f9fae19+gd81a15d 40 24/stable canonical✓ -
core24 20240426 405 latest/stable canonical✓ base
pc 24-0.1 178 24/stable canonical✓ gadget
pc-kernel 6.8.0-31.31 1803 24/stable canonical✓ kernel
snapd 2.62 21465 latest/stable canonical✓ snapd
maciek-borzecki@localhost:~$ sudo snap run --shell automount-core.daemon
root@localhost:/home/maciek-borzecki# cd $SNAP
root@localhost:/snap/automount-core/17# mkdir -p /media/auto/usb
root@localhost:/snap/automount-core/17# mount -t vfat /dev/sda2 /media/auto/usb -o rw,sync
root@localhost:/snap/automount-core/17# ls /media/auto/usb
EFI NvVars snaps systems
root@localhost:/snap/automount-core/17# umount /media/auto/usb
root@localhost:/snap/automount-core/17# ls /media/auto/usb
root@localhost:/snap/automount-core/17#
seems to work just fine AFAICT
# mkdir -p /media/auto/usb
# mount -t vfat /dev/sda /media/auto/usb -o rw,sync
mount: /media/auto/usb: permission denied.
dmesg(1) may have more information after failed mount system call.
The dmesg output:
[25749.789889] usb 1-1.3: new high-speed USB device number 15 using dwc2
[25749.890582] usb 1-1.3: New USB device found, idVendor=048d, idProduct=1234, bcdDevice= 2.00
[25749.890625] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[25749.890648] usb 1-1.3: Product: Disk 2.0
[25749.890667] usb 1-1.3: Manufacturer: USB
[25749.890685] usb 1-1.3: SerialNumber: 92070970DE275658798
[25749.893085] usb-storage 1-1.3:1.0: USB Mass Storage device detected
[25749.898483] scsi host0: usb-storage 1-1.3:1.0
Can you try to run the mount command inside the snap shell again but set LIBMOUNT_DEBUG=all in the environment? Eg:
LIBMOUNT_DEBUG=all mount -t vfat /dev/sda /media/auto/usb -o rw,sync
and also please run lsblk outside of the snap environment and attach the output.
Here it is
# LIBMOUNT_DEBUG=all mount -t vfat /dev/sda /media/auto/usb -o rw,sync
2705: libmount: INIT: library debug mask: 0xffffff
2705: libmount: INIT: library version: 2.39.3
2705: libmount: INIT: feature: selinux
2705: libmount: INIT: feature: smack
2705: libmount: INIT: feature: btrfs
2705: libmount: INIT: feature: verity
2705: libmount: INIT: feature: namespaces
2705: libmount: INIT: feature: idmapping
2705: libmount: INIT: feature: statx
2705: libmount: INIT: feature: assert
2705: libmount: INIT: feature: debug
Available "LIBMOUNT_DEBUG=<name>[,...]|<mask>" debug masks:
all [0xffffff] : info about all subsystems
cache [0x000004] : paths and tags cache
cxt [0x000200] : library context (handler)
diff [0x000400] : mountinfo changes tracking
fs [0x000040] : FS abstraction
help [0x000001] : this help
hook [0x008000] : hooks functionality
locks [0x000010] : mtab and utab locking
loop [0x002000] : loop devices routines
options [0x000008] : mount options parsing
optlist [0x010000] : mount options container
tab [0x000020] : fstab, mtab, mountinfo routines
update [0x000080] : mtab, utab updates
utils [0x000100] : misc library utils
monitor [0x000800] : mount tables monitor
btrfs [0x001000] : btrfs specific routines
verity [0x004000] : verity specific routines
2705: libmount: CXT: [0xaaab1a169880]: ----> allocate
2705: libmount: OPTLIST: [0xaaab1a1699e0]: alloc
2705: libmount: OPTLIST: [0xaaab1a1699e0]: registr map 0xffff9a2ee960
2705: libmount: OPTLIST: [0xaaab1a1699e0]: registr map 0xffff9a2eec00
2705: libmount: OPTLIST: [0xaaab1a1699e0]: append rw,sync
2705: libmount: OPTLIST: [0xaaab1a1699e0]: added rw [id=0x00000001 map=0xffff9a2ee960]
2705: libmount: OPTLIST: [0xaaab1a1699e0]: added sync [id=0x00000010 map=0xffff9a2ee960]
2705: libmount: FS: [0xaaab1a169c40]: alloc
2705: libmount: CXT: [0xaaab1a169880]: mount: preparing
2705: libmount: CXT: [0xaaab1a169880]: use default optsmode
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000010 [map=0xffff9a2ee960]
2705: libmount: CXT: [0xaaab1a169880]: OPTSMODE (file-part): force=0, fstab=1, mtab=1
2705: libmount: CXT: [0xaaab1a169880]: fstab not required -- skip
2705: libmount: OPTLIST: [0xaaab1a1699e0]: merging
2705: libmount: CXT: [0xaaab1a169880]: mount: evaluating permissions
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000000 [map=0xffff9a2eec00]
2705: libmount: CXT: [0xaaab1a169880]: perms: superuser [rc=0]
2705: libmount: CXT: [0xaaab1a169880]: --> preparing options
2705: libmount: CXT: [0xaaab1a169880]: ---> stage:prep-options
2705: libmount: CXT: [0xaaab1a169880]: calling __selinux [first]
2705: libmount: HOOK: [0xffff9a2ef720]: SELinux fix options
2705: libmount: CXT: [0xaaab1a169880]: calling __idmap [first]
2705: libmount: CXT: [0xaaab1a169880]: calling __owner [first]
2705: libmount: CXT: [0xaaab1a169880]: <--- stage:prep-options [rc=0 status=1]
2705: libmount: CXT: [0xaaab1a169880]: <-- preparing options done [rc=0]
2705: libmount: CXT: [0xaaab1a169880]: --> preparing source path
2705: libmount: CXT: [0xaaab1a169880]: srcpath '/dev/sda'
2705: libmount: CACHE: [0xaaab1a169d90]: alloc
2705: libmount: CACHE: [0xaaab1a169d90]: canonicalize path /dev/sda
2705: libmount: CACHE: [0xaaab1a169d90]: add entry [ 1] (path): /dev/sda: /dev/sda
2705: libmount: CXT: [0xaaab1a169880]: ---> stage:prep-source
2705: libmount: CXT: [0xaaab1a169880]: calling __loopdev [first]
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000000 [map=0xffff9a2eec00]
2705: libmount: CXT: [0xaaab1a169880]: calling __veritydev [first]
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000000 [map=0xffff9a2eec00]
2705: libmount: CXT: [0xaaab1a169880]: <--- stage:prep-source [rc=0 status=1]
2705: libmount: CXT: [0xaaab1a169880]: final srcpath '/dev/sda'
2705: libmount: CXT: [0xaaab1a169880]: --> preparing fstype
2705: libmount: CXT: [0xaaab1a169880]: FS type: vfat [rc=0]
2705: libmount: CXT: [0xaaab1a169880]: --> preparing target path
2705: libmount: CACHE: [0xaaab1a169d90]: canonicalize path /media/auto/usb
2705: libmount: CACHE: [0xaaab1a169d90]: add entry [ 2] (path): /media/auto/usb: /media/auto/usb
2705: libmount: CXT: [0xaaab1a169880]: ---> stage:prep-target
2705: libmount: CXT: [0xaaab1a169880]: calling __mkdir [first]
2705: libmount: CXT: [0xaaab1a169880]: calling __subdir [first]
2705: libmount: CXT: [0xaaab1a169880]: <--- stage:prep-target [rc=0 status=1]
2705: libmount: CXT: [0xaaab1a169880]: final target '/media/auto/usb' [rc=0]
2705: libmount: CXT: [0xaaab1a169880]: checking for helper
2705: libmount: CXT: [0xaaab1a169880]: /sbin/mount.vfat ... not found
2705: libmount: CXT: [0xaaab1a169880]: /sbin/fs.d/mount.vfat ... not found
2705: libmount: CXT: [0xaaab1a169880]: /sbin/fs/mount.vfat ... not found
2705: libmount: CXT: [0xaaab1a169880]: ---> stage:prep
2705: libmount: CXT: [0xaaab1a169880]: calling __legacy-mount [first]
2705: libmount: CXT: [0xaaab1a169880]: appending mount hook from __legacy-mount
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000010 [map=0xffff9a2ee960]
2705: libmount: CXT: [0xaaab1a169880]: <--- stage:prep [rc=0 status=1]
2705: libmount: CXT: [0xaaab1a169880]: --> prepare update
2705: libmount: CXT: [0xaaab1a169880]: utab path initialized to: /run/mount/utab
2705: libmount: CXT: [0xaaab1a169880]: checking for writable tab files
2705: libmount: UTILS: utab: /run/mount/utab
2705: libmount: UTILS: try write /run/mount/utab dir: (null)
2705: libmount: UTILS: access OK
2705: libmount: UPDATE: [0xaaab1a16aa70]: allocate
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000010 [map=0xffff9a2ee960]
2705: libmount: UPDATE: [0xaaab1a16aa70]: resetting FS [target=(null), flags=0x00000010]
2705: libmount: UPDATE: [0xaaab1a16aa70]: FS template:
2705: libmount: UPDATE: 2705: libmount: FS: [0xaaab1a169c40]: synced: vfs: 'rw,sync' fs: '(null)' user: '(null)', optstr: 'rw,sync'
------ fs:
source: /dev/sda
target: /media/auto/usb
fstype: vfat
optstr: rw,sync
VFS-optstr: rw,sync
2705: libmount: UPDATE: prepare utab entry
2705: libmount: UPDATE: utab entry unnecessary (no options)
2705: libmount: CXT: [0xaaab1a169880]: mount: do mount
2705: libmount: CXT: [0xaaab1a169880]: ---> stage:pre-mount
2705: libmount: CXT: [0xaaab1a169880]: <--- stage:pre-mount [rc=0 status=1]
2705: libmount: CXT: [0xaaab1a169880]: ---> stage:mount
2705: libmount: CXT: [0xaaab1a169880]: calling __legacy-mount [active]
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000010 [map=0xffff9a2ee960]
2705: libmount: HOOK: [0xffff9a2ef670]: mount(2) [source=/dev/sda, target=/media/auto/usb, type=vfat, flags=0x00000010, options=<none>]
2705: libmount: HOOK: [0xffff9a2ef670]: mount(2) failed [errno=1 Operation not permitted]
2705: libmount: CXT: [0xaaab1a169880]: <--- stage:mount [rc=1 status=-1]
2705: libmount: CXT: [0xaaab1a169880]: mnt_context_do_mount() done [rc=1]
2705: libmount: HOOK: [0xffff9a2ef780]: deinit '__loopdev'
2705: libmount: HOOK: [0xffff9a2ef760]: deinit '__veritydev'
2705: libmount: HOOK: [0xffff9a2ef740]: deinit '__mkdir'
2705: libmount: HOOK: [0xffff9a2ef720]: deinit '__selinux'
2705: libmount: HOOK: [0xffff9a2ef700]: deinit '__subdir'
2705: libmount: HOOK: [0xffff9a2ef670]: deinit '__legacy-mount'
2705: libmount: CXT: [0xaaab1a169880]: removing mount hook from __legacy-mount
2705: libmount: HOOK: [0xffff9a2ef6e0]: deinit '__idmap'
2705: libmount: HOOK: [0xffff9a2ef6c0]: deinit '__owner'
2705: libmount: CXT: [0xaaab1a169880]: mnt_context_mount() done [rc=1]
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000010 [map=0xffff9a2ee960]
2705: libmount: OPTLIST: [0xaaab1a1699e0]: return flags 0x00000000 [map=0xffff9a2eec00]
2705: libmount: CXT: [0xaaab1a169880]: excode: rc=32 message="permission denied"
mount: /media/auto/usb: permission denied.
dmesg(1) may have more information after failed mount system call.
2705: libmount: CXT: [0xaaab1a169880]: <---- reset [status=0] ---->
2705: libmount: FS: [0xaaab1a169c40]: free [refcount=0]
2705: libmount: OPTLIST: [0xaaab1a1699e0]: remove rw
2705: libmount: OPTLIST: [0xaaab1a1699e0]: remove sync
2705: libmount: CACHE: [0xaaab1a169d90]: free [refcount=0]
2705: libmount: UPDATE: [0xaaab1a16aa70]: free
2705: libmount: CXT: [0xaaab1a169880]: Setting (null) as target namespace
2705: libmount: CXT: [0xaaab1a169880]: free
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 61.2M 1 loop /snap/core24/410
/run/mnt/base
/
loop1 7:1 0 11.9M 1 loop /snap/pi/142
/run/mnt/gadget
loop2 7:2 0 149.3M 1 loop /snap/pi-kernel/825
/run/mnt/kernel-snaps/pi-kernel/825
/run/mnt/kernel
/usr/lib/modules
/usr/lib/firmware
loop4 7:4 0 460K 1 loop /snap/automount-core/18
loop6 7:6 0 10.3M 1 loop /snap/console-conf/41
loop12 7:12 0 33.7M 1 loop /usr/lib/snapd
/snap/snapd/21467
sda 8:0 1 7.5G 0 disk
mmcblk0 179:0 0 29G 0 disk
├─mmcblk0p1 179:1 0 1.2G 0 part /var/lib/snapd/seed
│ /boot/piboot
│ /run/mnt/ubuntu-seed
├─mmcblk0p2 179:2 0 750M 0 part /run/mnt/ubuntu-boot
├─mmcblk0p3 179:3 0 32M 0 part /var/lib/snapd/save
│ /run/mnt/ubuntu-save
└─mmcblk0p4 179:4 0 27G 0 part /var/snap
/var/tmp
/var/log
/var/lib/systemd
/var/lib/snapd
/var/lib/private/systemd
/var/lib/misc
/var/lib/extrausers
/var/lib/dbus
/var/lib/console-conf
/var/lib/cloud
/var/cache/snapd
/var/cache/apparmor
/snap
/root
/home
/run/mnt/data
/etc/polkit-1/rules.d
/etc/security/pwquality.conf
/etc/update-motd.d
/etc/machine-id
/etc/environment
/etc/default/swapfile
/etc/sysctl.d
/etc/cloud
/etc/udev/rules.d
/etc/systemd
/etc/sudoers.d
/etc/ssh
/etc/network/if-up.d
/etc/netplan
/etc/modules-load.d
/etc/modprobe.d
/etc/locale.conf
/etc/iproute2
/etc/hosts
/etc/dbus-1
/etc/writable
/writable
Thank you. I think I know what is going on. Can you file a bug in snapd LP project for this?
We have attempted to tighten the sandbox in core24, and snaps using core24 as base will now be placed in a mandatory device cgroup. This wasn’t the case before, i.e. a snap which did not have any devices assigned to it would not have been placed in device cgroup. This behavior is only triggered when using core24+ base, such that snaps using earlier bases continue to work as expected.
With the new behavior, all device access is mediated, but it’s clear that we have hit a regression with the mount-control interface. Specifically, it will continue to work when using snapctl command to create mounts, as those are effectively set up as mount units and executed by systemd (transient or persistent). However, a direct call to mount() is checked against device access permissions, which ends up returning EPERM most likley at this location https://elixir.bootlin.com/linux/v6.8.10/source/block/bdev.c#L834. Now, since the snap was switched to core24 base, it now gets a mandatory device cgroup, and so only this set of devices is allowed by default:
$ PATH=$PATH:$PWD ./tests.device-cgroup automount-core.daemon dump
c 139:* rwm
c 10:239 rwm
c 5:2 rwm
c 141:* rwm
c 138:* rwm
c 1:7 rwm
c 142:* rwm
c 143:* rwm
c 140:* rwm
c 10:200 rwm
c 136:* rwm
c 5:0 rwm
c 1:9 rwm
c 1:3 rwm
c 137:* rwm
c 1:5 rwm
c 1:8 rwm
c 5:1 rwm
The list obviously does not include any b 8:<n> which would correspond to /dev/sd* nodes. The mount-control interface does not synthesize rules for udev to tag devices for the snap, and so the hits EPERM.
For now, I would recommend one of the following steps:
- keep core22 as the base
- request custom-device interface with rule /dev/sd*, and udev tags for the relevant subsystem
- use snapctl to create mounts
In the meantime I’ll try to put a task to address this in the upcoming sprint.