Wayland Screenshots in Electron Snap (ubuntu-frame - mir)

alibmarbi · September 20, 2024, 3:21am

Hi all,

I’m developing a digital signage app with Electron, packaged as a Snap, running as a daemon under Ubuntu Frame using Wayland (on Ubuntu Server 22.04 and 24.04).

The app has two main goals:

Load and display content (HTML, CSS, JS) from a URL.
Take screenshots of the visible window for remote monitoring.

Goal 1 is working well. However, for Goal 2, I encountered issues. Initially, I used Electron’s DesktopCapturer but found it only supports X11 for screenshots.

Since my app runs on Wayland, I explored alternatives and discovered the ubuntu-frame.screenshot tool, which uses Grim for taking screenshots.

I successfully captured screenshots by running this command manually on terminal (required sudo).

To integrate this tool with my Electron app, I created a bash script and executed it via Node’s child process. This worked only in --devmode, which isn’t suitable for publishing.

I attempted to replicate the ubuntu-frame.screenshot setup in my Snap as my-snap.screenshot, but it also required --devmode to function. Interestingly, I had to enable snap set ubuntu-frame config="add-wayland-extensions=zwlr_screencopy_manager_v1" for my tool, but not for the original (ubuntu-frame.screenshot). Not sure why

Now, I can take Wayland screenshots, but I’m unsure how to integrate this into my Electron Snap.

I’m considering packaging my screenshot tool as a Debian service to listen for D-Bus messages, and try to send requests from my Electron snap to it, but decided to ask here before proceeding.

Has anyone faced similar issues? Any simpler solutions to integrate Wayland screenshots with Electron would be appreciated!

Thanks for your time!

alibmarbi · September 20, 2024, 3:26am

Hi @alan_g and @Saviq,

I hope you don’t mind me tagging you directly. I noticed your involvement in Mir (Ubuntu Frame) development and thought you might have some insights on my issue.

Thank you in advance for your help!

Saviq · September 20, 2024, 6:38am

You needed sudo because the Wayland service (Frame) runs as root, so you need that to access its socket at /run/user/0/wayland-0.

That sounds like a missing interface connection. snappy-debug can help pinpointing what accesses are missing.

That’s because the ubuntu-frame family of snaps is privileged to access those, sensitive, protocol extensions:

github.com

canonical/ubuntu-frame/blob/b034b221f137d4f2ac49ed25e909e7aca458cd89/src/frame_authorization.cpp#L25-L41


      
          AuthModel const auth_model{{
              {"ubuntu-frame-osk", {
                  WaylandExtensions::zwlr_layer_shell_v1,
                  WaylandExtensions::zwp_virtual_keyboard_manager_v1,
                  WaylandExtensions::zwp_input_method_manager_v2,
              }},
              {"ubuntu-frame-vnc", {
                  WaylandExtensions::zwlr_screencopy_manager_v1,
                  WaylandExtensions::zwlr_virtual_pointer_manager_v1,
                  WaylandExtensions::zwp_virtual_keyboard_manager_v1,
              }},
              {"ubuntu-frame", {
                  WaylandExtensions::zwlr_screencopy_manager_v1,
                  WaylandExtensions::zwlr_layer_shell_v1,
                  WaylandExtensions::zwlr_foreign_toplevel_manager_v1
              }},
          }};

The problem with enabling the screencopy protocol like you did is that it’s enabled globally - anyone that has access to the socket (aka, is connected to the wayland interface and runs as the same user) can take screenshots, at any time. So you do it at your own risk.

EDIT: I’ve filed this feature request on Frame:

github.com/canonical/ubuntu-frame

Allow privileged snaps to access sensitive protocols

opened 06:59AM - 20 Sep 24 UTC

Saviq

enhancement triaged

There are valid reasons to allow external snaps to e.g. capture screenshots, or …provide input methods. It's currently only possible to globally enable those protocols, opening up for attacks. We need a way to allow privileged (e.g. through snap interface connections) snaps to access those, and for the device operator to decide. Applying apparmor labels is what we always thought can be the mediation layer here.

Now, that really depends on your needs. If you want to take them periodically, you could use a daemon: oneshot service with a timer: defined. If you need to react to some external trigger, that’s something you’ll need to roll on your own. And how will you access the grabbed screenshots? The possibilities are endless here, really - and whatever works for you is good.

alibmarbi · September 23, 2024, 7:39am

Hi @Saviq (thanks for the help and support)

Since my Electron app is running as a daemon, I thought it would already have the correct “permissions” to spawn a node child process using sudo, but it seems I’m not allowed to (not sure why yet), but I will give it another try and see if I have more info using the snappy-debug as you mentioned.

Got it, that makes sense and thanks for raising the feature request, do let me know if any more information to support the request is needed from me and I’d be happy to provide any help to improve that.

Yeah, for now, to buy me some time, I’m able to periodically take screenshots and save it on a folder that my Electron app can read (SNAP_USER_COMMON) and then synchronize it whatever needed inside my Electron app.

However, I will keep researching alternatives to only take screenshots when needed within Electron (snap) instead of another service running at the same time.