I think we understand the issue now.
The snapd 2.28.4 release should fix this particular problem.
You can try it out by refreshing to the beta channel of the core snap:
sudo snap refresh core --beta
Please report any issues you find while running beta on the forum!
I’m having the same issue on > 20 systems (part of our CI system for frr). It started approx 3 days ago.
Tried to upgrade to 2.28.4, but does not make a difference.
The interesting part I can see is that I only see after I connect the needed interfaces for my application
(frr snap is the one I’m supporting).
root@ci-comp2-dut:~# snap install --beta frr
frr (beta) 3.0-rc2 from 'osr' installed
root@ci-comp2-dut:~# frr.vtysh
Hello, this is FRRouting (version 3.0-rc2).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
ci-comp2-dut# exit
root@ci-comp2-dut:~# snap connect frr:network-control core:network-control
root@ci-comp2-dut:~# frr.vtysh
udev_enumerate_scan failed
root@ci-comp2-dut:~#
This is Ubuntu 16.04 with hwe kernel, updated to latest (It started to fail before, but I’ve tried to upgrade to latest packages etc trying to solve the issue without any success)
All of my Ubuntu 16.04 are consistent with the error - every time and reboot etc doesn’t seem to work.
(I can provide access to systems with this issue if there is any interest)
@mwinter - When looking at the nvidia issue I noticed network-control was also affected and this is fixed in https://github.com/snapcore/snapd/pull/4031 which will be in 2.28.5 (hopefully available very soon in the beta channel).
Great. I hope this is release soon.
And I think this might be the right fix. Looking at the pull request, I’ve noticed the discussion on tun/tap interfaces and my system have a tap0 interface. For a test, I’ve deleted the tap0 and this fixes the issue
An annoying side effect of the force upgrade policy on snap’s ( See Disabling automatic refresh for snap from store thread - I still wish a manual lock to version would be possible and I could lock the core as well )
@mwinter - until the fix is out, you can either revert to 2.27.6 of the snap or you can edit /etc/udev/rules.d/70-snap.frr.rules and remove the following lines:
KERNEL=="tap[0-9]*", TAG+="snap_frr_vtysh"
KERNEL=="tun[0-9]*", TAG+="snap_frr_vtysh"
Then run:
$ sudo udevadm trigger
We apologize for the inconvenience.
I might also mention that we’ll be adding regression tests for this so it doesn’t happen again going forward.
I think you may also need udevadm control --reload-rules before the trigger command.
We also found that:
/run/udev/snap_*/*nvidia* (at least on xenial, it doesn’t seem to affect artful)1 is already fixed in master and in the release branch, 2 is under review.
We pushed a new core to the beta channel that should fix this error. We tracked it down to an issue with the nvidia module and udev/sysfs. Please try snap refresh --beta core if you see this issue. After the refresh it should go away.
I have been testing 2.28.5 (via core in beta channel) and it’s resolved the issues for me. Thanks for fixing these tricky issues.
This just bit me as well. Refreshing core to beta seems to have solved it.
Still not resolved for me. My snap requires the raw-usb interface. Tested on core stable and beta:
stable: 16-2.28.5
beta: 16-2.29~rc1
This was not an issue before. Whenever I disconnect the raw-usb interface, the error disappears. I have refreshed core several times now. Removed and installed my custom app several times. To no affect.
raw-usb works for me:
$ snap interfaces | grep raw-usb
:raw-usb test-policy-app
$ grep raw /var/lib/snapd/apparmor/profiles/snap.test-policy-app.raw-usb
...
profile "snap.test-policy-app.raw-usb" (attach_disconnected) {
# Description: Allow raw access to all connected USB devices.
$ test-policy-app.raw-usb -c 'cat /run/udev/data/+usb:2-0:1.0'
I:1416103
E:ID_USB_CLASS_FROM_DATABASE=Hub
E:ID_VENDOR_FROM_DATABASE=Linux Foundation
E:ID_MODEL_FROM_DATABASE=3.0 root hub
G:snap_test-policy-app_raw-usb
Can you open a new topic with more details? Please include:
snap version
journalctl |grep audit (just for entries corresponding to your last failing run)same here, I’m on 2.29~rc2 and with raw-usb I get same problem once interface is connected.
system is Ubuntu Core16
I just tried on Ubuntu Core 16 in a vm with the core from the beta channel (2.29), and it works fine:
$ snap version
snap 2.29
snapd 2.29
series 16
kernel 4.4.0-96-generic
$ snap interfaces | grep raw-usb
:raw-usb test-policy-app
$ grep raw /var/lib/snapd/apparmor/profiles/snap.test-policy-app.raw-usb
...
profile "snap.test-policy-app.raw-usb" (attach_disconnected) {
# Description: Allow raw access to all connected USB devices.
$ test-policy-app.raw-usb -c 'cat /run/udev/data/+usb:2-0:1.0'
I:2041395
E:ID_USB_CLASS_FROM_DATABASE=Hub
E:ID_USB_PROTOCOL_FROM_DATABASE=Full speed (or root) hub
E:ID_VENDOR_FROM_DATABASE=Linux Foundation
E:ID_MODEL_FROM_DATABASE=1.1 root hub
E:net.ifnames=0
G:snap_test-policy-app_raw-usb
Can you provide more steps on how to reproduce in a VM?
Is there a USB device with a proprietary driver plugged in? (proprietary drivers aren’t allowed to use sysfs. This came up recently with the nvidia driver)
I have not managed to reproduce this in vm so far, I did reproduce it on actual Intel HW running UC16 using this snap
https://launchpad.net/~ondrak/+snap/openhab-master/+build/100066/+files/openhab_2.2.0-SNAPSHOT_amd64.snap
snap rums without problem, but once you connect raw-usb it will be fail.
It does not happen on classic.
There is no USB device plugged in, though I did pugged device before which I was testing.
Rebooting, reinstalling snap makes no difference. (No USB device plugged in)
$ grep raw /var/lib/snapd/apparmor/profiles/snap.openhab.openhab
# Description: Allow raw access to all connected USB devices.
All I can see on syslog is this:
localhost systemd-udevd[536]: Network interface NamePolicy= disabled on kernel command line, ignoring.
localhost udisks2.udisksd[2089]: (udisksd:2089): udisks-CRITICAL **: [2089]: Error creating directory /etc/udisks2: Read-only file system [udiskslinuxdrive.c:244, configuration_get_path()]
localhost kernel: [ 493.552905] intel_soc_dts_thermal: request_threaded_irq ret -22
localhost systemd-udevd[28344]: Process '/sbin/crda' failed with exit code 234.
localhost udisks2.udisksd[2089]: (udisksd:2089): udisks-CRITICAL **: [2089]: Error creating directory /etc/udisks2: Read-only file system [udiskslinuxdrive.c:244, configuration_get_path()]
localhost kernel: [ 524.027079] audit_printk_skb: 6 callbacks suppressed
localhost kernel: [ 524.027086] audit: type=1400 audit(1509475083.271:864): apparmor="DENIED" operation="create" profile="/usr/lib/snapd/snap-confine" pid=28407 comm="snap-confine" family="inet" sock_type="dgram" protocol=0 requested_mask="create" denied_mask="create"
I tested now different VM with usb connected, and that seems to work. So problem is not there always, seems to be related to underlying system
Does the system use proprietary drivers?
nope, running stock pc-kernel snap.
FYI, https://github.com/snapcore/snapd/pull/4164 and https://github.com/snapcore/snapd/pull/4165 (2.29)