Wayland Screenshots in Electron Snap (ubuntu-frame - mir)

alibmarbi · September 20, 2024, 3:21am

Hi all,

I’m developing a digital signage app with Electron, packaged as a Snap, running as a daemon under Ubuntu Frame using Wayland (on Ubuntu Server 22.04 and 24.04).

The app has two main goals:

Load and display content (HTML, CSS, JS) from a URL.
Take screenshots of the visible window for remote monitoring.

Goal 1 is working well. However, for Goal 2, I encountered issues. Initially, I used Electron’s DesktopCapturer but found it only supports X11 for screenshots.

Since my app runs on Wayland, I explored alternatives and discovered the ubuntu-frame.screenshot tool, which uses Grim for taking screenshots.

I successfully captured screenshots by running this command manually on terminal (required sudo).

To integrate this tool with my Electron app, I created a bash script and executed it via Node’s child process. This worked only in --devmode, which isn’t suitable for publishing.

I attempted to replicate the ubuntu-frame.screenshot setup in my Snap as my-snap.screenshot, but it also required --devmode to function. Interestingly, I had to enable snap set ubuntu-frame config="add-wayland-extensions=zwlr_screencopy_manager_v1" for my tool, but not for the original (ubuntu-frame.screenshot). Not sure why

Now, I can take Wayland screenshots, but I’m unsure how to integrate this into my Electron Snap.

I’m considering packaging my screenshot tool as a Debian service to listen for D-Bus messages, and try to send requests from my Electron snap to it, but decided to ask here before proceeding.

Has anyone faced similar issues? Any simpler solutions to integrate Wayland screenshots with Electron would be appreciated!

Thanks for your time!

alibmarbi · September 20, 2024, 3:26am

Hi @alan_g and @Saviq,

I hope you don’t mind me tagging you directly. I noticed your involvement in Mir (Ubuntu Frame) development and thought you might have some insights on my issue.

Thank you in advance for your help!

Saviq · September 20, 2024, 6:38am

You needed sudo because the Wayland service (Frame) runs as root, so you need that to access its socket at /run/user/0/wayland-0.

That sounds like a missing interface connection. snappy-debug can help pinpointing what accesses are missing.

That’s because the ubuntu-frame family of snaps is privileged to access those, sensitive, protocol extensions:

github.com

canonical/ubuntu-frame/blob/b034b221f137d4f2ac49ed25e909e7aca458cd89/src/frame_authorization.cpp#L25-L41


      
          AuthModel const auth_model{{
              {"ubuntu-frame-osk", {
                  WaylandExtensions::zwlr_layer_shell_v1,
                  WaylandExtensions::zwp_virtual_keyboard_manager_v1,
                  WaylandExtensions::zwp_input_method_manager_v2,
              }},
              {"ubuntu-frame-vnc", {
                  WaylandExtensions::zwlr_screencopy_manager_v1,
                  WaylandExtensions::zwlr_virtual_pointer_manager_v1,
                  WaylandExtensions::zwp_virtual_keyboard_manager_v1,
              }},
              {"ubuntu-frame", {
                  WaylandExtensions::zwlr_screencopy_manager_v1,
                  WaylandExtensions::zwlr_layer_shell_v1,
                  WaylandExtensions::zwlr_foreign_toplevel_manager_v1
              }},
          }};

The problem with enabling the screencopy protocol like you did is that it’s enabled globally - anyone that has access to the socket (aka, is connected to the wayland interface and runs as the same user) can take screenshots, at any time. So you do it at your own risk.

EDIT: I’ve filed this feature request on Frame:

github.com/canonical/ubuntu-frame

Allow privileged snaps to access sensitive protocols

opened 06:59AM - 20 Sep 24 UTC

Saviq

enhancement triaged

There are valid reasons to allow external snaps to e.g. capture screenshots, or …provide input methods. It's currently only possible to globally enable those protocols, opening up for attacks. We need a way to allow privileged (e.g. through snap interface connections) snaps to access those, and for the device operator to decide. Applying apparmor labels is what we always thought can be the mediation layer here.

Now, that really depends on your needs. If you want to take them periodically, you could use a daemon: oneshot service with a timer: defined. If you need to react to some external trigger, that’s something you’ll need to roll on your own. And how will you access the grabbed screenshots? The possibilities are endless here, really - and whatever works for you is good.

alibmarbi · September 23, 2024, 7:39am

Hi @Saviq (thanks for the help and support)

Since my Electron app is running as a daemon, I thought it would already have the correct “permissions” to spawn a node child process using sudo, but it seems I’m not allowed to (not sure why yet), but I will give it another try and see if I have more info using the snappy-debug as you mentioned.

Got it, that makes sense and thanks for raising the feature request, do let me know if any more information to support the request is needed from me and I’d be happy to provide any help to improve that.

Yeah, for now, to buy me some time, I’m able to periodically take screenshots and save it on a folder that my Electron app can read (SNAP_USER_COMMON) and then synchronize it whatever needed inside my Electron app.

However, I will keep researching alternatives to only take screenshots when needed within Electron (snap) instead of another service running at the same time.

NWDigital · May 1, 2025, 9:37pm

I am having basically the same issues and almost exactly same use case. I need to take screenshots for sanity sake so I know wether or not to reload the browser or not.

I started a snap, unlisted right now 'Install nwd-agent on Linux | Snap Store" for the purpose of remote monitoring of Touchscreen devices at my work. I have a web app I wrote that allows folks to see who’s signed in or out, an In Out Board, I have a website dedicated to the web app at https://www.ioblist.com/.

BTW, the nwd-agent snap runs as a daemon so it can send info to my server at a set interval.

I have tried many things to be able to take screenshots on Ubuntu Core and Ubuntu Desktop using my custom NWD-Agent snap. I’ve been able to get screenshots using ubuntu-frame.screenshot on Ubuntu Core 24, but that doesn’t seem work for Ubuntu Desktop even when using Ubuntu Frame. Also, all the screens I need to montor have the display rotated in the Ubuntu-Frame config orientation=left so I was getting strange screenshots. The image was small and only in the bottom right corner and the image also had transparent pixels. To fix this, I incorporated ImageMagick into the snap and I am able to manipulate the screenshot taken by ubuntu-frame.screenshot and send that to my server. While not perfect, it’s better than the original. All this works fine in devmode, but as soon as I enable strict confinement --dangerous, screenshots no longer work, for the same reason stated by @Saviq.

This is only the start to my problems. I tried using dbus commands to take a screenshot on Ubuntu Desktop 24 from within my snap and it fails no matter what mode --devmode or not. If I run it like snap run nwd-agent, i get the following error: Error: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.portal.Desktop was not provided by any .service files and if I run the command like this sudo snap run nwd-agent i get a different error: Error connecting: Error spawning command line “dbus-launch --autolaunch=be84ce5e866a4a6286970bd9a4cbf4c0 --binary-syntax --close-stderr”: Child process exited with code 1.

So, basically, right now, I can only take screenshot from within my snap on Ubuntu Core in devmode, on Ubuntu Desktop, only way is to just not use the app at all and use imagemagick and a script to send screenshots to my server.

I’d love to have it all packaged up in one snap, but I have no idea what I need to do to make that happen. I’m new to making snaps so I’m learning fast, but hoping someone can point me in the right direction. For sanity sake

Oh, almost forgot, here’s the dbus command I use, it works outside of my snap on Ubuntu Desktop 24 but not when packaged in my snap.

`secret=$(echo $RANDOM | md5sum | head -c 20; echo)

var=$(gdbus call --session
–dest=org.freedesktop.portal.Desktop
–object-path=/org/freedesktop/portal/desktop
–method=org.freedesktop.portal.Screenshot.Screenshot
string:root “{‘handle_token’: <‘screenshot_${secret}’>, ‘modal’: , ‘interactive’: }”)`

Saviq · May 19, 2025, 3:31pm

Hi @NWDigital, sorry for the late reply, was away for a few weeks.

That’s a bug, filed here: `ubuntu-frame.screenshot` doesn't work on classic · Issue #224 · canonical/ubuntu-frame · GitHub

Another bug… Screenshots wrong when display rotated (again) · Issue #3909 · canonical/mir · GitHub

We’ll try and get those two fixed asap.

If it runs as daemon, that’s b/c it’s running outside of your user session.

The appropriate way would be fixing this:

github.com/canonical/ubuntu-frame

Allow privileged snaps to access sensitive protocols

opened 06:59AM - 20 Sep 24 UTC

Saviq

enhancement triaged

There are valid reasons to allow external snaps to e.g. capture screenshots, or …provide input methods. It's currently only possible to globally enable those protocols, opening up for attacks. We need a way to allow privileged (e.g. through snap interface connections) snaps to access those, and for the device operator to decide. Applying apparmor labels is what we always thought can be the mediation layer here.

It’s not trivial (which is why we have not yet tackled it), but tractable. The first thing would be to add an appropriate interface definition (e.g. ‘screen-recording’ or such) here:

Then, Frame could check for the appropriate AppArmor label to allow access to the privileged interfaces.