HotPlug implementation plan


#1

UPDATED 2018/04/17 after discussing some high-level ideas with Gustavo; most important change is the removal of event filters.

This is a high-level proposal of the HotPlug implementation (for USB initially), as a followup to the general plan discussed during last sprint and documented here: Development sprint March 5th, 2018 .

Prior to writing the notes below I did some experimentation with go-udev package (https://github.com/pilebones/go-udev) and it seems to be a nice candidate for doing the low-level bits for us. The main factor when looking for suitable udev-handling package for go was no reliance on cgo (we want a native Go implementation).

High level overview

New “UDev Monitor” in snapd will be responsible for monitoring for udev hotplug events and enumerating already existing devices on start. It will internally use go-udev or similar (but will hide some of its implementation details) to discover devices, and will also collect some extra attributes (such as idProduct and idVendor) when new device is reported, along with data normally present in uevent. UDev Monitor will report all hotplug events (devices added/removed) and ifacemgr will pass them down to all interfaces that implement the needed hotplug methods.

Reported events will be handled by ifacemgr. One way of doing this would be via tasks created by UDev monitor and handled by ifacemgr. Events reported by UDev Monitor will be passed to all interfaces that implement the needed hotplug API method. It’s perfectly fine for an event to be handled by more than one interface - for example, plugging a device results in read-only access via foo interface and write-access via foo-control interface.

Interfaces interested in hotplug devices will need to implement the following new method:

  • HotPlugDeviceKey: method that generates a unique key for given device, based on the data provided by UDev Monitor (e.g. a combination of productId + vendorId + serial); we will provide a default implementation of device key computation, but interfaces will be allowed to provide their own. The key will be used to recognize devices as they are unplugged and plugged again.
  • HotPlugDeviceAdded: method that receives hotplug event data (wrapped in a convinient object) as well as a “hotplug backend” object. The method decides if particular device is relevant for this particular interface and if so, produces a definition of a slot using the backend object.

Hotplug devices repository

Interface Manager will maintain a list of hotplug devices that were handled by interfaces - every supported device will have an entry in the state: its unique device key, list of slots and list of connections that persists even if device is unplugged), for example:

"hotplug": {
  "<devicekey>" : {
      "slots": [ <slot refs> ],
      "conns": {
         "<connref>": true,
      },
  }
}

Note: the hotplug conns here are independent from existing global conns structure we have right now. When a hotplug device has connections, the connection appears both here in hotplug conns and in global conns. If it’s disconnected with snap disconnect ..., it disappears from both structures. If however the device is unplugged while connection was active, the connection is only removed from global conns and remains in hotplug conns.
This hotplug map will obviously need to be kept in sync with installed snaps - if snap gets removed, it disappears from hotplug conns. We will however never remove devices from it, as we have no way of knowing if the device ever comes back.

Sample flow
A sample execution flow based on the above plan will look as follows:

  1. Camera is plugged to the USB port.
  2. UDevManager receives uevent, creates a new “hotplug-add” task and puts all uevent data in it (device name, env data) and idProduct, idVendor, serial., e.g.
{
   “device”: “/sys/devices/pci0000:00/0000:00:14.0/usb1/1-11/1-11:1.0/video4linux/video0”,
   “env”: {"MAJOR":"81" "MINOR":"0" "DEVNAME":"video0" "SUBSYSTEM":"video4linux"},
   “idProduct”: “57c3”, “idVendor”: “0bda”, “serial”: “200901010001”
}
  1. The “hotplug-task” is picked by ifacemgr handler,
  2. ifacemgr passes device data to all interfaces that implement HotPlugDeviceAdded method; default device key is computed unless the interface provides own HotPlugDeviceKey method.
  3. The implementation of “camera” interface creates slot defintion.
  4. The slot gets added the interfaces repository.
  5. Interface Manager consults its internal hotplug devices repository to see if the device was already known based on its device key and if so, the previous name of the slot is used. If the slots were connected, re-create the connections and run interface hooks.

Implementation steps

I’ve broken the implementation into steps with the intent of being able to do the work incrementally in a series of PRs; this is just a rough idea and the order and details are not neccessarily very precise.

  1. UDevMonitor as a new entity in snapd. Enumerates all existing devices and monitors for new ones. Doesn’t do anything about them just yet.
  2. Defintion of new interface methods (HotPlugDeviceAdded, HotPlugDeviceKey…), definition of the structs passed to these methods (udev event data, “backend” object to act on when adding new slot).
  3. Implementation of hotplug device repository in ifacemgr.
  4. Implementation of method that creates device key based on all attributes of the deivice
  5. Stubs for “hotplug-add” and “hotplug-remove” handlers.
  6. Implement “hotplug-add” task handler of ifacemgr: call the right interfaces, create slots on the fly.
    Insert slots into the repository.
  7. Implement “hotplug-remove” task handler of ifacemgr: disconnect hotplug slots.
  8. Autoconnect slots on hotplug where appropriate.

CC @niemeyer


List and table presentation ideas
Classic confinement for `subsurface`
Request to review my test gadget
Call for testing: Subsurface
#2

@pstolowski thanks for the outlining the plan, it seems a well considered approach.

In the “High level overview” you mention collecting extra attributes. Can I just point of that the current serial-port interface already identified one other attribute that was required for filtering. In the following PR we added and attribute “usb-interface-number”: https://github.com/snapcore/snapd/pull/4040/files

Firstly, I hope we can keep that attribute available to filter against in the new implementation.

Secondly, if additional extra attributes are identified that someone would like to filter on, would this require changes to snapd itself to expose the attribute?

Thanks


#3

Not sure if this is an appropriate place to discuss the current hotplug experimental support in snapd edge, but I’m having some issues testing this out with a Raspberry Pi 3B+ running Ubuntu Core 18.

I have the following questions:

  1. How should one enable hotplug on a Ubuntu Core 18 system? Do I still need to set experimental.hotplug = true for the core snap? Do I set it for the system snap? Both? (I can’t seem to set it for the snapd snap, it won’t let me)
  2. What udev attributes are necessary for this to work? I have tried using hotplug serial-port for both the /dev/ttyAMA0 serial port on the Rpi3B+ for the bluetooth controller, as well as an Arduino Uno I connected which shows up as /dev/ACM0 initially. The devices have these udev attributes defined:

arduino:

$ udevadm info /dev/ttyACM0 
P: /devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2:1.0/tty/ttyACM0
N: ttyACM0
S: serial/by-id/usb-Arduino__www.arduino.cc__0043_95635333131351201230-if00
S: serial/by-path/platform-3f980000.usb-usb-0:1.2:1.0
E: DEVLINKS=/dev/serial/by-id/usb-Arduino__www.arduino.cc__0043_95635333131351201230-if00 /dev/serial/by-path/platform-3f980000.usb-usb-0:1.2:1.0
E: DEVNAME=/dev/ttyACM0
E: DEVPATH=/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2:1.0/tty/ttyACM0
E: ID_BUS=usb
E: ID_MODEL=0043
E: ID_MODEL_ENC=0043
E: ID_MODEL_FROM_DATABASE=Uno R3 (CDC ACM)
E: ID_MODEL_ID=0043
E: ID_PATH=platform-3f980000.usb-usb-0:1.2:1.0
E: ID_PATH_TAG=platform-3f980000_usb-usb-0_1_2_1_0
E: ID_REVISION=0001
E: ID_SERIAL=Arduino__www.arduino.cc__0043_95635333131351201230
E: ID_SERIAL_SHORT=95635333131351201230
E: ID_TYPE=generic
E: ID_USB_CLASS_FROM_DATABASE=Communications
E: ID_USB_DRIVER=cdc_acm
E: ID_USB_INTERFACES=:020201:0a0000:
E: ID_USB_INTERFACE_NUM=00
E: ID_VENDOR=Arduino__www.arduino.cc_
E: ID_VENDOR_ENC=Arduino\x20\x28www.arduino.cc\x29
E: ID_VENDOR_FROM_DATABASE=Arduino SA
E: ID_VENDOR_ID=2341
E: MAJOR=166
E: MINOR=0
E: SUBSYSTEM=tty
E: TAGS=:systemd:
E: USEC_INITIALIZED=7561993964
E: net.ifnames=0

embedded bluetooth adapter:

$ udevadm info /dev/ttyAMA0 
P: /devices/platform/soc/3f201000.serial/tty/ttyAMA0
N: ttyAMA0
E: DEVNAME=/dev/ttyAMA0
E: DEVPATH=/devices/platform/soc/3f201000.serial/tty/ttyAMA0
E: MAJOR=204
E: MINOR=64
E: SUBSYSTEM=tty
E: TAGS=:systemd:
E: USEC_INITIALIZED=3895597
E: net.ifnames=0

#4

You’re right, something is amiss, I’ve just flashed pi3 with core18 and tested.

Yes, the feature is still experimental and can be enabled with either snap set core experimental.hotplug=true or snap set system ...., however neither has an effect, hotplug subsystem is not enabled at all for some reason.

As for the above devices, /dev/ttyACM0 should work, it has all the neccessary attributes (model, name, serial…). The /dev/ttyAMA0 wouldn’t work.

I’m currently debugging the problem with core18. Thanks for reporting!


#5

Ok, mystery solved… The problem is that for some reason we didn’t land serial-port interface changes for hotplug in the 2.38 release branch (at the time of release), it only landed in master, which means that even hotplug machinery is there, it has nothing to do… The 2.39 release wil have serial-pore code changes that make it hotplug-aware. Sorry about that :frowning:


#6

I switched to edge channels for core, core18, and snapd snap is that not sufficient for the hotplug aware serial-port interface?


#7

Ok, I actually found a bug related to core18+snapd transition that breaks it anyway… This will unfortunately need a fix.


#8

@pstolowski has this issue been already fixed?


#9

Yes, that was fixed.