Request for daemon + browser-support for krellian-kiosk

ijohnson · February 3, 2021, 9:01pm

@benfrancis to be clear, you are dropping user privileges after XWayland is launched, your script that calls setpriv with your electron app is executed after the xwayland-kiosk-launch script runs.

ijohnson · February 3, 2021, 9:04pm

Yes that should be fine, but you would also want this code in a post-refresh hook in order to ensure that users who have your snap currently installed are refreshed properly and continue to work (not sure if you have any such users right now). If you have no such existing userbase to support, then the install hook is perfectly fine.

Your logs don’t seem to imply that xwayland-kiosk-launch is to fault here, in fact I see that on the every execution the drop-snap-daemon.sh script starts running which would only be possible if the xwayland-kiosk-launch script finished successfully

Also reading through the logs on why the drop-snap-daemon.sh script failed in subsequent operations, perhaps you just need to change the script to something like

#!/bin/bash -ex
# setup a $SNAP_USER_DATA under $SNAP_DATA for snap-daemon user
mkdir -p $SNAP_DATA/snap-daemon-home/.config
chown  snap_daemon:snap_daemon $SNAP_DATA/snap-daemon-home
chown  snap_daemon:snap_daemon $SNAP_DATA/snap-daemon-home/.config

# set XDG_CONFIG_HOME to use our $SNAP_USER_DATA 
XDG_CONFIG_HOME=$SNAP_DATA/snap-daemon-home/.config 
export XDG_CONFIG_HOME

# maybe this is unnecessary, but just to be safe if electron needs $HOME for anything else
HOME=$SNAP_DATA/snap-daemon-home
export HOME

exec $SNAP/... # the existing command you have

this just removes the -R, so it doesn’t try to change permissions for subdirectories below there if they are differently privileged.

benfrancis · February 4, 2021, 9:00pm

Quick update:

The script no longer produces errors, thanks.

I managed to get past the “No protocol specified” error and segfault by giving the snap_daemon user access to the X server using xhost, as discussed in IRC.

I now have a new error from Electron:

Feb 04 20:54:27 Unknown-b8-27-eb-f2-7b-0b-2 krellian-kiosk[27595]: Cannot open pixbuf loader module file '/root/snap/krellian-kiosk/common/.cache/gdk-pixbuf-loaders.cache': Permission denied
                                                                   
                                                                   This likely means that your installation is broken.
                                                                   Try running the command
                                                                     gdk-pixbuf-query-loaders > /root/snap/krellian-kiosk/common/.cache/gdk-pixbuf-loaders.cache
                                                                   to make things work again for the time being.

I’m not sure why Electron is trying to access files in the /root directory…

# ls -al /root/snap/krellian-kiosk/common/.cache/
total 24
drwx------ 5 root root 4096 Feb  4 20:57 .
drwxr-xr-x 3 root root 4096 Jan 22 20:23 ..
drwxr-xr-x 2 root root 4096 Feb  4 18:58 fontconfig
-rw-r--r-- 1 root root 3721 Feb  4 20:57 gdk-pixbuf-loaders.cache
drwxr-xr-x 2 root root 4096 Feb  4 20:57 gio-modules
drwxr-xr-x 2 root root 4096 Feb  4 20:57 immodules

ijohnson · February 4, 2021, 9:16pm

This directory is $SNAP_USER_COMMON. I assume that one of the various wrapper scripts, possibly the xwayland-kiosk-launch one, is setting some environment variable to be $SNAP_USER_COMMON/.cache and you will want to change that to something like $SNAP_DATA/snap-daemon-home/.cache which will be owned/readable by the snap-daemon user.

ogra · February 4, 2021, 9:23pm

gdk-pixbuf-loaders.cache surely comes from a desktop-launch wrapper … unlikely to be xwayland-kiosk-launch … the desktop-launch wrappers are not really designed for kiosks, they usually need some “on the fly love” i.e. you probably also want something like this:

github.com

ogra1/kodi-pi-standalone/blob/master/snap/snapcraft.yaml#L133


  source-subdir: glib-only
  plugin: make
  build-packages:
    - libglib2.0-dev
  stage-packages:
    - libglib2.0-bin
  override-prime: |
    snapcraftctl prime
    # force armhf
    sed -i 's/export SNAP_LAUNCHER/export ARCH="arm-linux-gnueabihf"\nexport SNAP_LAUNCHER/' bin/desktop-launch
    # do not try to update xdg links in home (HOME is SNAP_COMMON)
    sed -i '/# Create links for user-dirs.dirs/,+22 s/^/#/' bin/desktop-launch
libcec:
  plugin: cmake
  after: [ pi-libs ]
  source: https://github.com/Pulse-Eight/libcec.git
  cmake-parameters:
    - -DRPI_INCLUDE_DIR=${SNAPCRAFT_STAGE}/usr/include
    - -DRPI_LIB_DIR=${SNAPCRAFT_STAGE}/usr/lib
    - -DCMAKE_PREFIX_PATH=${SNAPCRAFT_PART_INSTALL}/usr
    - -DCMAKE_INSTALL_PREFIX=/usr

to not have it attempt to create symlinks for “Music, Videos, Documents…”

benfrancis · February 5, 2021, 7:17pm

Yep, it looks like the desktop-launch wrapper is setting $XDG_CACHE_HOME to $SNAP_USER_COMMON/.cache and $GDK_PIXBUF_MODULE_FILE to $XDG_CACHE_HOME/gdk-pixbuf-loaders.cache, which Electron is then trying to access. See https://github.com/ubuntu/snapcraft-desktop-helpers/blob/master/common/desktop-exports#L391

I’m trying to go through and reset these environment variables in my script to locations that the snap_daemon user can access, moving any existing data across and setting the necessary permissions, but it’s proving challenging (there are 23 exports in that one file).

I’m guessing nobody has tried to use the desktop-launch wrapper around an application started as the snap_daemon user before, as this all worked as root.

Oh wow, I hope not. That looks… brittle!

Will keep trying next week…

benfrancis · February 11, 2021, 8:29pm

OK, drop-snap-daemon.sh has grown a bit (and some of this should probably be in the install hook), but this seems to get past the previous errors:

#!/bin/bash -ex

# Create data directories needed by Electron
mkdir -p $SNAP_DATA/snap-daemon-home
mkdir -p $SNAP_DATA/snap-daemon-home/.config
if [[ ! -d $SNAP_DATA/snap-daemon-home/.cache ]]
then
  # Copy .cache directory to somewhere snap_daemon can access
  cp -r $SNAP_USER_COMMON/.cache $SNAP_DATA/snap-daemon-home/
fi

# Give snap_daemon ownership over data directories
chown snap_daemon:snap_daemon $SNAP_DATA/snap-daemon-home
chown snap_daemon:snap_daemon $SNAP_DATA/snap-daemon-home/.config
chown -R snap_daemon:snap_daemon $SNAP_DATA/snap-daemon-home/.cache

# Tell Electron where to find data directories
export HOME=$SNAP_DATA/snap-daemon-home
export XDG_CONFIG_HOME=$SNAP_DATA/snap-daemon-home/.config
export XDG_CACHE_HOME=$SNAP_DATA/snap-daemon-home/.cache
export GDK_PIXBUF_MODULE_FILE=$XDG_CACHE_HOME/gdk-pixbuf-loaders.cache

# Give the snap_daemon user access to the X server
xhost si:localuser:snap_daemon

# Execute the rest of the command chain as the snap_daemon user
exec "$SNAP/usr/bin/setpriv" --clear-groups --reuid snap_daemon --regid snap_daemon -- "$@"

But xwayland-kiosk-launch/Electron still seems to be crashing:

Feb 11 20:15:19 Unknown-b8-27-eb-f2-7b-0b-2 krellian-kiosk.krellian-kiosk[1917]: /snap/krellian-kiosk/x1/bin/xwayland-kiosk-launch: line 151: 2075 Trace/breakpoint trap "$@"

Unfortunately I can’t see a helpful error message so I’m a bit stuck as to where to look next.

Full output here.

Any suggestion?

ijohnson · February 11, 2021, 8:45pm

This seems to be a crash from your Electron/JS code, are there any electron verbose / debug env vars or options you can turn on?

benfrancis · February 15, 2021, 7:26pm

Adding --enable-logging to the Electron application launch command reveals one additional error which I think is coming from Electron/Chromium:

Feb 12 20:05:13 Unknown-b8-27-eb-f2-7b-0b-2 krellian-kiosk.krellian-kiosk[3369]: [3540:0212/200513.750312:ERROR:edid_parser.cc(102)] Too short EDID data: manufacturer id

I’ve so far not been able to identify the cause of this error, but Googling around suggestions have included upgrading Chromium (I tried bumping my Electron version) and running Chromium with --disable-gpu (I don’t know whether Ubuntu Core supports hardware acceleration for Electron, but if it does I definitely don’t want to be disabling it), neither of which helped.

ijohnson · February 22, 2021, 9:42pm

Unfortunately this is outside my perview, but I would say that if we have any electron experts here who could help debug this they should try and help out, but orthogonal to that, perhaps we should just grant @benfrancis permission to use browser-support with daemon here as it seems to be proving actually quite difficult and not at all a simple change as was originally expected almost 9 months ago now.

benfrancis · February 23, 2021, 2:21pm

Heh, interesting timing.

I had my friend Sam help me with this and he managed to get my application to run as the snap_daemon user by adding a bunch of hacks to drop-snap-daemon.sh which copy a lot of things that the desktop-launch helper creates into locations the snap-daemon user can access, and resets various environment variables to point to them. You can see the full combined patch here.

There are a few things we’re still not sure about:

The actual crash appeared to be caused by the kiosk application trying to create a LevelDB database file in its own directory which is a read-only area of the filesystem. In order to work around this Sam added cd $SNAP_DAEMON_HOME to the end of drop-snap-daemon.sh so that the application is executing in a directory where it has write access. Is this a good solution or is there a better approach?
You’ll see that in drop-snap-daemon.sh there are quite a lot of things which need to be done on every start of the application. Could any of these be avoided by fixing things in Snapcraft Desktop Helpers or Ubuntu Core itself? Or could some things be moved to a hook like the install hook?
The hack to get the socket-activated service starting on launch is currently in the install hook, but doesn’t seem to work. Maybe because I currently need to manually connect plugs after installation? It actually needs to run on every launch not just install time. Above it was suggested that I need to move this somewhere like $SNAP/bin/activate-svc.sh and create a separate app in snapcraft.yaml as described above, but how do I then get that to start on launch as well?
The kiosk application is now fully working, but AppArmor is still complaining about some things:
1. Xwayland trying to access /proc/<krellian-kiosk PID>/cmdline. We don’t know why Xwayland needs this but might be due to the exec "$@" in desktop-launch and drop-snap-daemon.sh ?
2. Blocking trying to talk to various dbus services. This might down to some weirdness in the node-dbus package which is trying to do more than just access NetworkManager (which is what is used by the kiosk application)

I also see the following errors in the console:

Feb 23 14:12:18 Unknown-b8-27-eb-f2-7b-0b-2 krellian-kiosk.krellian-kiosk[5276]: [5498:0223/141218.563329:ERROR:bus.cc(393)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
Feb 23 14:12:19 Unknown-b8-27-eb-f2-7b-0b-2 krellian-kiosk.krellian-kiosk[5276]: [5498:0223/141219.005996:ERROR:bus.cc(393)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")

Although the application is now working, overall this adds up to quite a long list of hacks needed to get the application running as a non-root user, in addition to the hacks I already had to do in snapcraft.yaml to work around bugs in the snapcraft build system (e.g. 1886858 and 1886861).

I’m a bit concerned about how brittle this setup is going to be in the long term. If nothing else I think we’ve uncovered a bunch of things which could be improved in the platform.

Given the desktop-launch helper clearly isn’t designed for use with kiosk applications running as the snap_daemon user, perhaps there’s a need for a “kiosk-launch” helper (or possibly making xwayland-kiosk-launch work without desktop-launch) to make these kinds of things easier?

Update: I’ve noticed that one of the lines in drop-snap-daemon.sh fails on subsequent installs and causes the application to get stuck crashing again. It’s a permission error which we’re a bit perplexed by as that part of the script should be running as root.

Feb 23 15:28:13 Unknown-b8-27-eb-f2-7b-0b-2 krellian-kiosk.krellian-kiosk[18230]: + cp /root/snap/krellian-kiosk/x2/.last_revision /var/snap/krellian-kiosk/x2/snap-daemon-home/.last_revision
Feb 23 15:28:13 Unknown-b8-27-eb-f2-7b-0b-2 krellian-kiosk.krellian-kiosk[18230]: cp: cannot create regular file '/var/snap/krellian-kiosk/x2/snap-daemon-home/.last_revision': Permission denied

ijohnson · February 23, 2021, 10:52pm

This is a good solution, it’s why I originally had you setup this env var for the snap_daemon user

I imagine we will want to take most of this script and put it into a kiosk/ubuntu-core specific helper indeed, so all this work can be reused by others. I’m not sure who should take up the mantle on making this script more generic, perhaps some kiosk folks here can comment on who should be responsible there.

This additional service would not include the listen-streams, so it is not an activated service and it just starts up normally like any other daemon.

Can you show the full denial for this one and the dbus one below? I think the network-manager one is probably harmless, lots of snaps seem to have this denial for some reason…

benfrancis:

Feb 23 14:12:18 Unknown-b8-27-eb-f2-7b-0b-2 krellian-kiosk.krellian-kiosk[5276]: [5498:0223/141218.563329:ERROR:bus.cc(393)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")

Is this a chromium/electron error? A bit unclear to me what is failing here, but again if it’s not detrimental to your application maybe you can ignore this…

we ran into this above, see:

benfrancis · February 25, 2021, 5:21pm

OK, great I’ve added this as an additional simple daemon.

Feb 25 16:43:44 Unknown-b8-27-eb-f2-7b-0b audit[2245]: AVC apparmor="DENIED" operation="open" profile="snap.krellian-kiosk.krellian-kiosk" name="/proc/2313/cmdline" pid=2245 comm="Xwayland" requested_mask="r" denied_mask="r" fsuid=0 ouid=584788
Feb 25 16:43:44 Unknown-b8-27-eb-f2-7b-0b kernel: audit: type=1400 audit(1614271424.118:462): apparmor="DENIED" operation="open" profile="snap.krellian-kiosk.krellian-kiosk" name="/proc/2313/cmdline" pid=2245 comm="Xwayland" requested_mask="r" denied_mask="r" fsuid=0 ouid=584788


Feb 25 16:54:40 Unknown-b8-27-eb-f2-7b-0b audit[1365]: USER_AVC pid=1365 uid=100 auid=4294967295 ses=4294967295 msg='apparmor="DENIED" operation="dbus_signal"  bus="system" path="/fi/w1/wpa_supplicant1/Interfaces/1" interface="org.freedesktop.DBus.Properties" member="PropertiesChanged" name=":1.3" mask="receive" pid=2269 label="snap.krellian-kiosk.krellian-kiosk" peer_pid=1391 peer_label="unconfined"
Feb 25 16:54:40 Unknown-b8-27-eb-f2-7b-0b audit[1365]: USER_AVC pid=1365 uid=100 auid=4294967295 ses=4294967295 msg='apparmor="DENIED" operation="dbus_signal"  bus="system" path="/fi/w1/wpa_supplicant1/Interfaces/1" interface="fi.w1.wpa_supplicant1.Interface" member="PropertiesChanged" name=":1.3" mask="receive" pid=2269 label="snap.krellian-kiosk.krellian-kiosk" peer_pid=1391 peer_label="unconfined"
Feb 25 16:54:40 Unknown-b8-27-eb-f2-7b-0b kernel: audit: type=1107 audit(1614272080.594:490): pid=1365 uid=100 auid=4294967295 ses=4294967295 msg='apparmor="DENIED" operation="dbus_signal"  bus="system" path="/fi/w1/wpa_supplicant1/Interfaces/1" interface="fi.w1.wpa_supplicant1.Interface" member="PropertiesChanged" name=":1.3" mask="receive" pid=2269 label="snap.krellian-kiosk.krellian-kiosk" peer_pid=1391 peer_label="unconfined"

There may be others buried in there too.

I haven’t been able to figure out the cause of this error, but it certainly seems related to Chromium. Possibly something to do with Chromium running on the i3 window manager. It doesn’t seem to cause any obvious problems.

OK, Sam managed to fix this but then ran into a bunch of other problems which he also managed to fix. drop-snap-daemon.sh is now even longer. You can see our latest combined patch here.

I think we’re about there, though I wish this could all be less hacky. Hopefully some improvements to the platform might come out of this work.

If I clean up this patch and land it, do you think the snap might pass store review? I can then push out an installable release to the edge channel (I have a few more features to implement before a 0.1 beta/candidate/stable release).

Many many thanks for everyone’s help over an extended period of time! Particularly @ijohnson, @ogra and Sam.

alexmurray · February 26, 2021, 2:17am

FYI - The review-tools are not smart enough to know that the snap is dropping to the snap_daemon user etc so the krellian-kiosk snap will still require an override within the store to allow it to use both browser-support as a daemon - however, now that the snap appears to be properly supporting dropping to the snap_daemon user instead of running as root, I would be happy to vote in favor of granting the use of these combined accesses.

@advocacy - in this case, can you please perform publisher vetting (if this has not been done already previously… given how long this thread is now I can’t easily see if this has been done before)

Igor · February 26, 2021, 2:32pm

Yes, +1 from me, and I’ve verified the publisher. I worked with Ben before, and this is essential functionality for his appliance.

emitorino · March 12, 2021, 12:42pm

Since a review-tools override must be applied to allow this snap to use browser-support with daemon, I have manually approved the use of browser-support with daemon to krellian-kiosk until the review-tools changes are in place and this is done automatically. I already created the MP to support this and has been approved today, so it could probably be available on the store next week (cc @roadmr).

I also granted a declaration to allow krellian-kiosk to connect to itself via the x11 slot since it was required (cc @alexmurray @Igor @ijohnson).

This is now live.

benfrancis · March 12, 2021, 1:03pm

Thank you! Install Krellian Kiosk on Linux | Snap Store