[cross-post] chromium-mir-kiosk in portrait mode (rotated mir-kiosk layout)


#1

This post was intended as an answer to this thread over at discourse.ubuntu.com, but as a new user (at least over there) I would have to throw out a lot of links & mentions. Since this might be interesting to ppl here as well, I’m posting the full version here and will put a link in the forum there. Admins: Feel free to move if needed.

@greyback As a follow-up to our IRC discussion, here are my findings with the current chromium-mir-kiosk snap when rotating the mir-kiosk screen via miral-kiosk.display orientation property. My use-case is a smart mirror appliance which requires the display to be rotated 90° to Portrait mode. For the current version, I can assume a fixed resolution of 1080x1920 px.

Setup:

  • Ubuntu Core edge from 20181023 (tested on ubuntu-core-vm and Raspberry Pi 3B attached to HDMI display)
  • Installed mir-kiosk (stable) and chromium-mir-kiosk (beta)

Issue

When the mir display is rotated to the left as described, the chromium window rotates as well – but the i3 WM does not seem to pick up on it. Thus, all fullscreen windows are still in landscape mode, where the right half of the window goes off the screen. The following screenshot illustrates the issue:

This is the default behavior with no additional configuration changes apart from the rotation.

Workaround attempts

As @greyback mentioned, it seems that the i3 window manager does not pick up the rotation and thinks it’s in landscape orientation.

To experiment with config files in the chromium snap without rebuilding the snap, I cloned the source, adapted line 17 of snapcraft.yaml it so that the i3 configuration is loaded from $SNAP_DATA. Added cp $SNAP/etc/i3.config $SNAP_DATA/i3.config in snap/hooks/install to have the file available in a writable location. This way, you can edit the config with sudo editor /var/snap/chromium-mir-kiosk/current/i3.config and apply changes with snap restart chromium-mir-kiosk.

First attempt is to resize the window manually after disabling the fullscreen command:

# in $SNAP_DATA/i3.config
#for_window [class="^chromium$"] fullscreen
for_window [class="^chromium$"] resize set 1080 1920

This didn’t work at first. Reason: chromium-mir-kiosk starts chromium with a kiosk Chrome extension loaded by default – once disabled with snap set chromium-mir-kiosk disable kiosk=true, chromium is no longer in fullscreen mode and will thus be resized by i3.config commands. Unfortunately, this also means that you can see the i3 window title bar as well as the regular Chromium UI.
Adapting the snap’s glue/chromium-browser.launcher with a startup flag --kiosk has the same effect; it seems that chromium in kiosk mode will always run fullscreen. To quickly test command line flags (list of available flags), you can also install chromium on your regular host OS and run it with chromium-browser --your --flags --here.

Second attempt: Do not use kiosk mode (i. e. disable it through snap set) and start Chromium with the required dimensions instead. These are the changes I made to chromium-mir-kiosk snap source files:

# git diff for glue/chrome-browser.launcher
@@ -80,7 +80,100 @@ fi
    --password-store=basic \
    -- ignore-gpu-blacklist --enable-native-gpu-memory-buffers \
    -- class=chromium \
-   -- profile-directory=Default $EXTRA_ARGS
+   --app="https://my-app.tld" \
+   -- profile-directory=Default $EXTRA_ARGS
# git diff for glue/i3.config
@@ -7,8 +7,7 @@ 
font pango:monospace 8

# make chromium always start in fullscreen (chromium-browser --class chromium)
-for_window [class="^chromium$] fullscreen
+for_window [class="^chromium$] floating enable, resize set 1080 1920, move position 0 0, border none 

I also adapted snap/hooks/install to set the defaults as required (disablekiosk=true to prevent fullscreen behavior).

Configured this way, the complete browser window is rotated and visible on screen apart from a few pixels on the “top” of the window (i. e. left)

Questions

This is a hacky workaround, of course. It assumes a fixed resolution and discards all the benefits of kiosk mode. So to solve this in a clean way which can be committed for the general snap, I think one needs to figure out if the rotation / resolution is passed to i3 correctly, and if not, how to do it. Some questions for @greyback / @alan_g or anyone proficient with this stack:

  • Where should I start to investigate whether Xwayland passes rotation/resolution information to i3 at all? (completely new to graphics stacks, so any hint is helpful)
  • Related: How would I best debug this? Found the snap graphics-debug-tools-bboozzoo but can’t get the bundled tools to work after connecting it to the x11-plug slot of chromium-mir-kiosk.
  • I briefly tried to include x11-xserver-utils as a stage-package in my modified chromium-kiosk snap to test if xrandr commands (e. g. xrandr -o left) bundled as a snap command would achieve proper rotation for the X window. This was something @ogra suggested in IRC. All attempts failed with “could not open display”, as $DISPLAY does not seem to be set. I also tried to connect display-sounding interfaces from core to the snap, but to no avail. Am I doing it completely wrong, or is this a path worth pursuing?

Thank you in advance for all helpful answers!

Best,
Tobias


#2

I can’t be much help. But somewhere in the snap there is a script that chooses a DISPLAY and starts Xwayland. Until that runs DISPLAY won’t be set.

HTH


#3

Had to focus on other issues, but returning to this now. As @ogra suggested, I tried simply removing the i3 window manager – which has absolutely zero effect on the mentioned issue. Without deeper understanding, it seems to me that the X server does not get proper rotation information from Xwayland. I also spent some time on the xrandr/$DISPLAY issue: As you mentioned, $DISPLAY should be set from this line – but when I add xrandr -o left in line 152, I still get “can’t open display”.

I’m quite lost here and would appreciate any help or hints. (shameless ping @greyback)


#4

This doesn’t help with the problem, but might deepen your understanding: Xwayland is the X server.

Xwayland is also the Wayland client that uses Wayland to connect to mir-kiosk. Other Wayland clients (e.g. those in mir-kiosk-apps) respond correctly to the display configuration messages sent by mir-kiosk.

This problem lies somewhere between Xwayland, the Window Manager (i3) and the X client (chromium). At least one of them is doing the wrong thing.


#5

Hi @tobias!
Did you manage to fix this issue “the right way”?
We have the same issue and probably we could collaborate in some way to solve this together?


#6

It may be unrelated, but during the testing of Mir 1.1 I’ve found an issue with the notification messages Mir generates for outputs. I’ll revisit this issue once I’ve got to the bottom of that.
[edit]
That problem turned out to be client-side: the Mir server is behaving correctly.


#7

Hi @ilya, sorry for the long delay! Had to attend to more urgent parts of our stack (being the sole developer has downsides … :slight_smile: ) and some family time.

I’m just returning to this issue. @alan_g 's clarification (thanks, helped me understand this a lot better!), The XWayland config applied in the chromium-mir-kiosk snap (specifically in xwayland-kiosk-helper) in the snap might be a good starting point to debug, especially if/why $DISPLAY is not set properly. I fiddled around with a modified copy of xwayland-kiosk-helper which prints the Mir port used to set $DISPLAY, and it came up empty. Don’t know if I did that right.

Side note: I discovered the Webkit Platform for Embedded (WPE) project which has native Wayland support and should be less resource-hungry (our project runs on RasPi-like boards). I hope that a native Wayland client browser simply adheres to Mir display commands, so I’m looking into snapping this instead. The build instructions are lacking (and not working on my Ubuntu Desktop installation) at best, but if you’re interested let me know.


#8

Please let us know if this works for you. (Or not.)


#9

@alan_g : I have a snap ready with wpewebkit, its required libs and including the WebKit minibrowser. It has core18 as base. There’s only one issue remaining, and I think it’s related to the wayland socket. Here’s what I’ve done so far:

  1. Built libwpe, wpebackend-fdo and wpe-webkit from the latest release tarballs on my Ubuntu 18.04 Desktop (Hint: dependencies cannot be satisfied with package versions from 16.04 repos, even though wpe-webkit‘s github README instructs to build on Xenial). wpe-webkit includes the MiniBrowser.
  2. Installed miral-kiosk and ran it according to the “make a Wayland-native kiosk“ tutorial.
  3. Ran the built MiniBrowser binary, showed up successfully in Mir-on-X window
  4. Recreated the build process as a snap with parts for each component as well as a launcher script
  5. Built said snap on amd64 with Multipass, installed it on my 18.04 host

Everything works well, except the MiniBrowser fails with “failed to create the WPE view Backend”. The same error occurs if I specify different wayland sockets for miral-kiosk and the hand-compiled MiniBrowser bin, so I guess it’s related to WPE not finding the socket. I followed the kiosk tutorial closely, especially the section about the socket path issue for $XDG_RUNTIME_DIR. FYI, the symlink command listed there (ln -s $XDG_RUNTIME_DIR/../wayland-0 $XDG_RUNTIME_DIR/) does not traverse the path up, but created a link to /run/user/1000/snap.webkit-embedded/../wayland-0, which does not exist. I tried to create a relative link with ln -sr ../wayland-0 $XDG_RUNTIME_DIR/. This resolves correctly when I follow the link, but I don’t know if that is correct – didn’t resolve the issue anyway.


#10

There are “known problems” with graphics snaps on Ubuntu Classic. In particular, mir-kiosk fails to supply the “wayland” interface. There is a fix “coming soon” that you can get by:

snap refresh --edge core

Does the above fix this?

I confess it’s a while since I looked at the tutorials, so I’m not clear on where in which one you find the above advice. Anyway, looking at one “client” snap I know works: https://github.com/MirServer/mir-kiosk-apps/blob/master/snap/snapcraft.yaml:

    environment:
      XDG_RUNTIME_DIR: /run/user/0

HTH


#11

@tobias I sought some clarification and while the extract I posted “works” it isn’t good practice, nor the way things should be.

What should be happening is that snapd should be creating $XDG_RUNTIME_DIR. But it isn’t.

Unfortunately, the client cannot use mkdir -p -m 700 $XDG_RUNTIME_DIR to create the directory (/run/user/0/snap.mir-kiosk-apps in the example I was trying) as there is no /run/user/0 either. UNLESS there is a server (mir-kiosk) already running (which will have created it).

A second problem is that the link command will fail unless the target exists. That also requires a server to be running.

If (and only if) there’s a server already running this works:

mkdir -p -m 700 $XDG_RUNTIME_DIR
ln -sf /run/user/0/wayland-0 $XDG_RUNTIME_DIR/

We are trying to clear this all up and provide a reliable mechanism. So, while the above advice is a pragmatic solution for now, be prepared for changes.


#12

@alan_g Thank you for taking the time to investigate this! I’ve updated my post above with links, in short:

I tested your suggestions, but I think I either missed/misunderstood something or the problem is indeed a missing library in the snap instead of confinement issues.

Question: Should I add either the environment line in the apps’ snapcraft.yaml section or the $XDG_RUNTIME_DIR linking on service startup – or both? I understood it as “environment works, but is not good practice, use the symlinking instead”.

Here’s what I tried:

  1. Start the miral-kiosk application like in the tutorial (runs as user 1000), no other parameters. Creates a wayland-0 socket in /run/user/1000
  2. Switch the snap definition from service to app (i.e. disable service stanzas in snapcraft.yaml) so that if I run the app, it’ll run as user 1000
  3. Add the symlink workaround from your last post to the launch script, but linking to /run/user/1000/wayland-0 where miral-kiosk is running

… no luck. Still, building and running the MiniBrowser with the exact same steps but on the native host opens the browser window in Mir-on-X without issues.

Second attempt:

  1. Install mir-kiosk and refresh core from edge channel
  2. mir-kiosk starts on VT4
  3. Switch snap definition back to a service (should run as user 0)
  4. Revert the symlink to point to /run/user/0/wayland-0
  5. Make sure mir-kiosk is still running on VT4, SSH from other machine and start the wpe-webkit-mir-kiosk service again

… still the same error: “Failed to create WPE view backend”.

I found a (not very helpful) Github Issue, but it’s in the context of the buildroot overlay which I presume does not have the whole graphics stack provided by Ubuntu Core / mir-kiosk.

Please do correct me, but I presume that if the MiniBrowser runs fine in Mir-on-X from the host 18.04 machine, it should run fine from within a core18-based snap?


#13

One obvious problem is in https://git.launchpad.net/wpe-webkit-snap/tree/src/launch-wpe

export LIBGL_DRIVERS_PATH=$SNAP/usr/lib/<replaced by SNAPCRAFT_ARCH_TRIPLET>/dri

That’s an invalid path. It is better to use the “environment” in the .yaml, There you can use SNAPCRAFT_ARCH_TRIPLET. Vis:

      LIBGL_DRIVERS_PATH: ${SNAP}/usr/lib/${SNAPCRAFT_ARCH_TRIPLET}/dri

#14

This is just a placeholder which gets replaced through a sed command in the override-pull step for this part, see https://git.launchpad.net/wpe-webkit-snap/tree/snap/snapcraft.yaml#n139.

But I guess it’s cleaner if defined in the environment.


#15

This is the current $XDG_RUNTIME_DIR situation, running mir-kiosk and my snap in service mode directly on a 18.04 host with edge core.

➜ sudo tree /run/user/0

/run/user/0
├── snap.wpe-webkit-mir-kiosk
│   └── wayland-0 -> /run/user/0/wayland-0
├── wayland-0
└── wayland-0.lock
➜ sudo ls -laR /run/user/0

/run/user/0:
total 0
drwxr-xr-x 3 root root 100 Mär  7 16:34 .
drwxr-xr-x 5 root root 100 Mär  7 09:47 ..
drwx------ 2 root root  60 Mär  7 16:34 snap.wpe-webkit-mir-kiosk
srw-rw-rw- 1 root root   0 Mär  7 16:34 wayland-0
-rw-r----- 1 root root   0 Mär  7 16:34 wayland-0.lock

/run/user/0/snap.wpe-webkit-mir-kiosk:
total 0
drwx------ 2 root root  60 Mär  7 16:34 .
drwxr-xr-x 3 root root 100 Mär  7 16:34 ..
lrwxrwxrwx 1 root root  21 Mär  7 16:34 wayland-0 -> /run/user/0/wayland-0
user@host:~$ snap version
snap    2.38~pre1+git1182.68177c3~ubuntu16.04.1
snapd   2.38~pre1+git1182.68177c3~ubuntu16.04.1
series  16
ubuntu  18.04
kernel  4.15.0-46-generic

And for reference, this is still the only traceable error (highlight by me):

user@host:~$ snap logs wpe-webkit-mir-kiosk
2019-03-07T15:43:59Z systemd[1]: Started Service for snap application wpe-webkit-mir-kiosk.browser.
# ----- ⬇︎
2019-03-07T15:44:00Z -[5235]: Failed to create WPE view backend
# ----- ⬆︎
2019-03-07T15:44:00Z systemd[1]: snap.wpe-webkit-mir-kiosk.browser.service: Main process exited, code=exited, status=1/FAILURE
2019-03-07T15:44:00Z systemd[1]: snap.wpe-webkit-mir-kiosk.browser.service: Failed with result 'exit-code'.
2019-03-07T15:44:00Z systemd[1]: snap.wpe-webkit-mir-kiosk.browser.service: Service hold-off time over, scheduling restart.
2019-03-07T15:44:00Z systemd[1]: snap.wpe-webkit-mir-kiosk.browser.service: Scheduled restart job, restart counter is at 5.
2019-03-07T15:44:00Z systemd[1]: Stopped Service for snap application wpe-webkit-mir-kiosk.browser.
2019-03-07T15:44:00Z systemd[1]: snap.wpe-webkit-mir-kiosk.browser.service: Start request repeated too quickly.
2019-03-07T15:44:00Z systemd[1]: snap.wpe-webkit-mir-kiosk.browser.service: Failed with result 'exit-code'.
2019-03-07T15:44:00Z systemd[1]: Failed to start Service for snap application wpe-webkit-mir-kiosk.browser.

#16

Yes, sorry. There’s nothing else leaps out at me: An error from <some app I don’t know> failing to <something else I don’t know> doesn’t give me much to go on.

You really need to debug your snap and see exactly what is going wrong. I’m no expert on the techniques involved.

There’s some guidance in the “common problems” sections of this tutorial: https://tutorials.ubuntu.com/tutorial/wayland-kiosk#0

And there is more material around here: Snapcraft Live 6th Feb 2019

Good luck!


#17

@alan_g I managed to get the snap running properly: on Ubuntu 18.04 LTS desktop both with miral-kiosk or mir-kiosk with edge core; on Ubuntu Core using mir-kiosk with stable core. I’ll outline the main steps below, for fellow snap crafters that may run into similar issues (or have the same lack of experience with graphic stacks as I had). So bear with me – or skip to “Questions” :slight_smile:

The main problem were missing/wrong XDG environment variables inside the snap env. After I adapted them from the mir-kiosk-apps example you linked, WPE could connect to the Wayland socket and the EGL display. Then some iterative “libxyz is missing”, and it worked on Ubuntu Desktop. I added plugs and the snapcraft-preload plugin recommended by snappy-debug, and proceeded to test the ARM build on Core.

I was confused at first why the WebKit process was stuck in a crash loop after I installed the snap on Core running on a RasPi 3B. When I removed snapcraft-preload from the startup command, it stopped crashing – but loading any trivial website had weird issues that indicated that the browser can’t find libraries despite LD_LIBRARY_PATH being set. Issues like no images, no SSL/TLS despite libglib-networking being present etc. Also, this was in devmode, where WPE Webkit may access /dev/shm (which I gathered snapcraft-preload provides a workaround for in strict mode).

The WPE WebKit backend libwpebackend-fdo relies on the FreeDesktop.org stack, so I added the gtk3-desktop helper. Now everything works fine, I added two basic configuration options (URL and a toggle for the Remote Web Inspector) as well.

Some links to WPE WebKit in general:
https://wpewebkit.org → main site
https://github.com/WebPlatformForEmbedded → GitHub org
https://github.com/Igalia/cog → Fullscreen browser launcher used in this snap


Questions

I added TODO / FIXME to snapcraft.yaml to indicate the work-in-progress parts. Most concern dependencies for building/running WPE Webkit, but I’d be stoked if someone can answer me these questions which I think are more related to snapcraft/Core/mir:

  1. From experience with other snaps, would a slimmer desktop-helper than gtk3 suffice to provide the “normal” FDO stack required by libwpebackend-fdo? Including this helper brings the snap from ~75 to ~114MB. (Also asked this the WPE maintainers: libwayland-client and libwayland-server may suffice)
  2. The browser fails to resolve zeroconf/Bonjour hostnames, though it works fine when building and running cog + WPE WebKit natively works fine. I included some avahi-related libs and connected the avahi-observe interface, but I guess I’m missing something fundamental here.

Meta

I’d like to suggest amending the docs for Debugging building snaps with a workflow that sped up my debugging on-device (ARM snap on a RasPi 3) tremendously:

  1. Let the ARM build run on Launchpad – takes ~ 3h for the WPE snap instead of forever (or probably failing due to limited CPU/RAM) when building on a RasPi
  2. Load the snap file from LP to your device, either via scp from your main machine or using @ondra 's great toolbox snap
  3. Run unsquashfs -d my-dir-name the-snap-file_0.1.0_somearch.snap
  4. You can now snap try my-dir-name (optionally with --devmode) and edit all files in my-dir-name to your heart’s content, all instantly reflected inside the snap. Only when editing my-dir-name/meta/snap.yaml to change app commands/services or add interfaces, you need to remove and re-mount the snap (via snap try).

Step 4 (snap try) is already documented there, but I think fleshing out an extra sentence how that works if you only have a .snap file (LP build or snapcraft 3 with bases) can help expediting dev workflows.


#18

Ping @ogra and @ilya – might be of interest for you (especially @ogra 's MagicMirror snap). In my tests, WPE WebKit adhered to all Wayland commands that mir-kiosk sent its way, so it’s fullscreen by default and in any rotation that you set in miral-kiosk.display.

The snap will be available as wpe-webkit-mir-kiosk for amd64 and armhf on the beta channel once these builds succeed. amd64 already available and tested with success :slight_smile:


#19

That’s great!

I’m in the middle of releasing Mir 1.1.2 and updating all our server snaps, but I’ll take a closer look at your snap after that.

Once you’re satisfied with “polishing” I think your work deserves a thread in its right - “chromium-mir-kiosk in portrait mode” hardly indicates what you have achived.

I see a “TODO” before your use of “layout:” but this is a perfectly sane use of the feature.

I’ve not done any work on “slimming down” the helper dependencies for snaps. Maybe someone with more experience can help here? @popey?

Sorry, no idea about this.

Let me repeat: Great work! You have reason to be proud.


#20

@tobias I also just wanted to say congrats. I’ve not had much time to fix the chromium-mir snap, so I’m glad you’ve found a solution that works for you. Thanks for sharing what you’ve learned too!
-G