Deploying a kiosk on Ubuntu-core

I am trying to deploy a kiosk using Ubuntu-core, but I am facing quite a few issues.
My application is built using vuejs and is running on a docker container on Ubuntu core.
I am facing issues while displaying the front-end of the application.

I have tried wpe-webkit-mir-kiosk and pointed it to localhost:8081(url of the front-end server),but wpe-webkit-mir-kiosk doesn’t play any video or audio (mp4 and mp3 format) in my app. I have tested it with YouTube and wpe-webkit -mir-kiosk has no issues with video or audio playback there.

After that I built a snap of mir-kiosk-chromium from a github repo and ran my app with it. The app functioned properly with it, but mir- kiosk-chromium was using a lot of cpu resources and at times the display didn’t update when page was changed on the app (i.e on changing to a different page the browser kept displaying the previous page, but all the functioning and the sound playing were of the target page).

Then I created a simple snap of an electron app that pointed to localhost:8081, but it also faced the same issue as mir-kiosk-chromium, so I created a complete electron app of my application’s frontend and snapped it, but again it ran into the same issue as mir-kiosk-chromium.

Another issue with all the above solutions was that the text was not being displayed properly. On all the chromium based solutions the text was shifted to the right a light and at places it overlapped and on webkit it was shifted down a bit.

I need to deploy this application on Ubuntu-core and I am out of ideas. Does anyone has any ideas of what else I can try or how I can resolve some of the above mentioned issues?

funny, i have just spent my weekend poking around in an electron kiosk “browser” to get something chromium based into the store as a demo for customers …

my focus was on the raspberry Pi though …

by default any chromium based solution will do SW rendering only for your video playback, for x86 it should be easy through some flags and switches for the chromium component to enable GPU accelerated video playback and i guess you’d have to bundle libvdpau or some such in your snap …

regarding your shifted font issues, i’d recommend simply using one of the gnome extensions to provide you with a proper fontconfig setup …

my work from the weekend is at:

it utilizes the gnome-3-38 extension to provide a semi-normal desktop environment setup (theme, font, icon support) … using it as-is (no HW accel for video playback) i get proper 1080p playback on youtube even though it eats 80% of the CPU cores for that … but there is no stuttering or delay in video or audio at least …

for full HW accel on a Pi it will need to use a patched ffmpeg with the MMAL patches, but as i said, i guess on x86 something like VDPAU bundling should actually work …

@tobias - have you any suggestions for the wpe-webkit-mir-kiosk difficulties?

Thank you for the input. I will look into your recommendations. Regarding the font issues, I am already using gnome 3.28 and set the following in my snapcraft.yml file:
layout:
/usr/share/fonts:
bind: $SNAP/usr/share/fonts
I also tried your electron kiosk snap and it has the same issue.
Do I need to do some other configurations too?

if you want the fonts from the extension, you should use

bind: $SNAP/gnome-platform/usr/share/fonts

which is where the gnome snap provides its content …
i do not think you need/can/should have to modify other settings in the app (at least i don’t have to when running on a pi4 or in an x86 qemu) … what hardware is this ?

I am using this machine link

okay, regarding your video acceleration, you might perhaps get away with shipping libva2 in your stage-packages and setting

app.commandLine.appendSwitch('enable-features', 'VaapiVideoDecoder');

(completely untested indeed :slight_smile: (might need “electron=latest” in your package.json))

EDIT: here is also a lit of possible options to play with: https://gist.github.com/Informatic/6a5597ed8321035074e2c9a5a2847371

Thank you very much for sharing the above information it was very helpful. I got hardware decoding to work on the electron app on Ubuntu 20 using electron 12 (on electron 13 hardware decoding got enabled but was not being utilized according to the dev tools), setting

app.commandLine.appendSwitch(‘enable-features’, ‘VaapiVideoDecoder’);

and installing the i965-va-driver-shaders, libva-drm2 and libva-x11-2.

I added the above three packages to stage -packages and tried out the snap (both your above mentioned project and mine) and tested on ubuntu-core 20 but hardware acceleration is still not being utilized (I also tried with libva2 and all the flags you shared above). The chrome://gpu page shows video-decode: hardware enabled, but when I check through dev tools the video is still being decoded with out hardware acceleration.

Also regarding the font issue your project already has bind: $SNAP/gnome-platform/usr/share/fonts, but it also displays the text improperly.

Any clue what else I can try?

well, do you see any change in the CPU utilization (i tend to use the htop snap for this, it works well on Ubuntu Core) when crome://gpu reports HW accel perhaps the dev tools are simply not reporting the right thing ?

Yes, I did check through htop and there was no change in CPU utilization.
I also installed Ubuntu 20 on the same machine and tested the application there and there was a significant decrease in CPU utilization (with dev tools showing hardware decoding being used). So this is the solution to my problem, but I just can’t seem to get it working on Ubuntu core.

sorry, missed this. @Ahtasham619 If you have a minute, could you please enable debug logging like this:

  • snap set wpe-webkit-mir-kiosk error-to-console=true
  • snap set wpe-webkit-mir-kiosk debug=true

Then retry your Vue app on localhost:8081 and check with snap logs -f wpe-webkit-mir-kiosk.daemon if any helpful message comes up? I suspect the codec may be missing or cannot be located.

Thanks for getting back to me. I tried the above and got quite a long log. I didn’t find anything particularly helpful in the log. I am attaching the log file here maybe you can find some indication of what is wrong? logs

The first indicator are ALSA/audio playback related errors. I have a local branch building that includes also-base and the corresponding layout; I’ll give you a ping once it is ready. How is the audio/video embedded in your app? Just a regular <audio> / <video> tag?

Yes, I am just using regular audio and video tags. Here are the exact tags

<video ref=“videoRef” id=“videoRef” class=“test_img” type=“video” autoplay loop :poster=“require(’…/assets/images/main.jpg’)” :src=“require(’…/assets/videos/test.mp4’)” >

<audio ref=“audio” :src=“recording” autoplay id=“audio” ></audio>