Rapid Photo Downloader confinement -- is classic the only option?

Hi,

Rapid Photo Downloader might need to be a classic snap, unless I’m missing something.

First, Rapid Photo Downloader needs to be able to not just respond to hotplug events, but to command the host system GVFS/GIO to eject (unmount) cameras & phones. Otherwise Rapid Photo Downloader’s libgphoto2 processes are blocked from accessing the camera/phones, and irate users rate Rapid Photo Downloader one star because “it cannot download from my phone”. (I came across this last year because GVFS/GIO changed the way it represents libgphoto2 ports, which broke my auto-unmount code).

Second, Rapid Photo Downloader needs to be able to open a file manager on the host system so that users can right click on a thumbnail and see the file in their file browser.

Third, it is desirable for Rapid Photo Downloader to be able to download (ingest) from to wherever the user chooses, not just home or /media. I know from bug reports that people do that (and I’m also one of them, to be honest). For example a windows partition or an NFS share might be mounted in a precise location.

Let me know your thoughts.

Thanks,
Damon

We currently do not allow manipulating gvfs/gio in this manner. The snap should typically instead plugs removable-media or possibly udisks2. Is there a specific reason why automount by udisks2 isn’t working for you?

You can plugs the desktop interface then use xdg-open /path/to/file/or/dir (be sure to use /usr/bin/xdg-open from the runtime and not from inside your snap).

This is not typically a justification for classic. The mountpoints may be in /media or /mnt when using removable-media.

If you haven’t seen it already and for criteria we’ve seen before in this forum, Process for reviewing classic confinement snaps lists supported and unsupported criteria for use of classic.

Before proceeding further, we might need to clarify some things. I’m unclear about a lot of things relating to snap, and most people do not fully understand the subtleties of libgphoto2.

For the purposes of the discussion here, by ‘libgphoto2 device’ I mean ‘camera or phone’. Furthermore, when I write “PTP” you can read “PTP or MTP”, and likewise libptp can be read as libptp or libmtp. (MTP is a superset of PTP).

I don’t think it makes sense to talk of relationship between the usdisks2 plug and the removable-media plug and a libgphoto2 device. If there is a relationship, what is it? A libgphoto2 device is not a disk and it’s not removable media. It’s a device accessed by PTP using libgphoto2. So the usdisks2 and removable-media plugs ought to be irrelevant when it comes to accessing libgphoto2 devices. (These plugs will be relevant when it comes to access to things like camera memory cards inserted via USB, I suppose).

libgphoto2 uses the process libptp to access devices. A libgphoto2 device can only be accessed by one process at a time, which has exclusive control over and access to the device until it relinquishes it. No other process can read it, including another libgphoto2 process.

That is why for any program on any desktop, irrespective of whether snap is running or not, gPhoto2 or Rapid Photo Downloader or any other program that uses libgphoto2 cannot access a libgphoto2 device if another process is already using it.

Under Gnome, as soon as you plug in a libgphoto2 device, Gnome immediately takes control of the libgphoto2 device using GIO/ GVFS. This cannot be changed.

Under KDE, if you browse a libgphoto2 device using a KDE program like dolphin, KDE retains control of the libgphoto2 device and as I recall does not relinquish it until the program is closed.

This is a long and complicated way of saying that if a program like gPhoto2 or Rapid Photo Downloader wants to access a libghoto2 device, Gnome and KDE must relinquish control or else gPhoto2 and Rapid Photo Downloader and every other program that use libgphoto2 will not work.

Fortunately Gnome has a mechanism to relinquish control: https://lazka.github.io/pgi-docs/#Gio-2.0/classes/Mount.html#Gio.Mount.unmount_with_operation

If I understand your statement correctly, I think you’re saying that snap confinement means that no confined program can make a valid call to Gio.Mount.unmount_with_operation under any circumstances because of security reasons? Is that correct?

If that is correct, that breaks programs that use libgphoto2. Users have to know they need to unmount / eject the libgphoto2 device in Gnome files, which very few users know. The users simply conclude the program does not work. That is why I spent a lot of time in Rapid Photo Downloader writing code that automatically calls GIO/GVFS for it to relinquish control of libgphoto2 devices.

I’m sorry but I don’t understand what this means: be sure to use /usr/bin/xdg-open from the runtime and not from inside your snap.

I had not seen it previously, but I have now thanks.

essentially it means, do not put xdg-utils (the package that ships xdg-open) into your stage-packages: anywhere in your snapcraft.yaml so that the one from the core/core18 snap is used.

I was thinking about devices that show up as usb-storage.

Thank you for this detailed explanation. It is super helpful.

I’m saying that today we don’t currently support this but that we might be able to which, thanks to the above information, we can investigate further how it could be supported. Sorry if that was unclear.

I’m guessing these gio APIs map to DBus calls in which case I suspect adding a new interface would be possible. I don’t have access to a ptp device (I can probably locate an mtp device though). What would be the simplest way to locally test the APIs you require (or may require going forward) using either a dummy device or an mtp device? This doesn’t have to be a snap (but could be; do you have a working devmode snap? Classic?). Perhaps there is a utility I can install from the Ubuntu archive to more or less emulate what your snap does?

Thank you for the clarification, now I understand it. I think I need to clarify something myself. Regarding opening a file in a file browser, I’m referring to the action where a user wants a file highlighted in the file browser (that is, they’re not opening the file itself), i.e.:
Open-in-file-manager

(Sorry forgot to include the mouse pointer in the screenshot)

As you might be aware, Linux file managers do not act the same way when being passed a file URI. Some of them require a special command line argument to be told to scroll to and select the files in the file manager (the obvious behavior), rather than simply launch / open them (why anyone would want to call a file manager simply to launch or open a file is puzzling to me). These command line switches include --select and --show-item (it varies depending on the file manager program).

AFAIK xdg-utils were not designed with this level of control in mind, making them inapplicable for this use case. Am I misunderstanding something?

Most smartphones support MTP so if you have access to one of those you’re good to go in terms of testing. If it runs Android >= 6.0 (and I guess IOS?) that’s even better as modern devices can only do file transfer when they’re unlocked and the user has explicitly enabled file transfer on the phone itself. And BTW simply enabling file transfer on the phone causes the device to be unmounted and remounted as if it’s a new device. Do this in Android 9:

I don’t know of any program that handles GVFS / GIO the way my program does. So I think it’s best to test using the actual program. I guess it would be possible to build a separate test program but to be honest that would be a fair bit of work and my time is limited.

My current efforts of building a snap of Rapid Photo Downloader have made some progress but for now the program faces some serious limitations even when in devmode. Nonetheless the snap does build and happily the program can be run.

The latest release of Rapid Photo Downloader is in Debian unstable (it was packaged yesterday). Or you can run it in a Python virtual environment or for your user using my install script (the script can uninstall the program itself and its dependencies too, when you’re done).

For the snap, sadly the only way to get it for now is to do a lightweight check out:

bzr checkout --lightweight bzr+ssh://bazaar.launchpad.net/+branch/rapid/
cd rapid
snapcraft

Whatever install method you use for the non-snap version, I strongly recommend using version 0.9.15 or newer as older versions do not properly handle changes made in GVFS/GIO handling of libgphoto2 devices.

The test procedure is this (under Gnome):

Phase 1

  1. Run Rapid Photo Downloader (not in a snap)
  2. Plug in a phone
  3. Rapid Photo Downloader will detect it, instruct GVFS/GIO to unmount it, but then complain file transfers and not enabled
  4. Enable file transfer on the phone
  5. Rapid Photo Downloader will detect it again, instruct GVFS/GIO to unmount it, and scan it for photos / videos

Test done PASS

Phase 2

  1. Plug in a phone
  2. Enable file transfer on the phone
  3. Run Rapid Photo Downloader (not in a snap)
  4. Rapid Photo Downloader will detect it, instruct GVFS/GIO to unmount it, and scan it
  5. Unplug the phone, Rapid Photo Downloader should handle that without errors

Test done PASS

Phase 3

  1. Run Rapid Photo Downloader from the snap in devmode
  2. Plug in a phone
  3. Nothing happens, as Rapid Photo Downloader did not receive the system call about the device insertion

Test done FAIL

Phase 4

  1. Plug in a phone
  2. Run Rapid Photo Downloader from the snap in devmode
  3. Rapid Photo Downloader will detect it, instruct GVFS/GIO to unmount it (which fails)
  4. Enable file transfer on the phone
  5. Rapid Photo Downloader will detect it, instruct GVFS/GIO to unmount it (which fails)

Test done FAIL

Phase 5

  1. Plug in a phone
  2. Enable file transfer on the phone
  3. Manually eject it from Gnome files
  4. Run Rapid Photo Downloader from the snap in devmode
  5. Rapid Photo Downloader will scan it for photos / videos

Test done PASS

Phase 6

  1. Plug in a phone
  2. Enable file transfer on the phone
  3. Manually eject it from Gnome files
  4. Run Rapid Photo Downloader from the snap in devmode
  5. Rapid Photo Downloader will scan it for photos / videos
  6. Remove the device
  7. Nothing happens, as Rapid Photo Downloader did not receive the system call about the device removal

Test done FAIL

If you’re interested in the actual system calls, they’re all in this Python module, including using GIO/GVFS and also udev / udisks2 (that code took plenty of work to produce). Maybe that code might be useful for your own testing should you want to develop your own scripts?

One final note: the first time generating thumbnails will be slow on phones, as Rapid Photo Downloader has to download the entire JPEG / DNG / HEIF to be able to render the thumbnail. It’s much faster with a real camera.

1 Like

Ok, I’ve added this to my todo to explore options for snapd 2.45. This is a fair chunk of unplanned work, so I’m not sure that it will be in snapd 2.45.

To answer your original question, yes, today, classic is the only option for you. It does sound like strict mode is possible though, with the investigative work I mentioned. I suggest keep working at the devmode snap. I’ll post any PRs in this topic for you to experiment with. If you are eager to get the snap out, we could issue classic on a temporary basis.