Limitation when writing to SNAP_USER_COMMON

Hello,

Today I submitted a classic confinement request for my app. When running in classic confinement, the application works normally and is able to write data to SNAP_USER_COMMON as expected.

In an attempt to use the home interface instead, I’m trying to run it in strict mode. Yet when I start the app, it throws errors, related to ElasticSearch not being able to write in SNAP_USER_COMMON:

java.lang.IllegalStateException: failed to obtain node locks, tried [[/home/pirhoo/snap/datashare/common/index]] with lock id [0]; maybe these locations are not writable or multiple nodes were started witho
ut increasing [node.max_local_storage_nodes] (was [1])?                                                                                                                                                            
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:302)                                                                                                                                  
        at org.elasticsearch.node.Node.<init>(Node.java:362)                                                                                                                                                       
        at org.icij.datashare.text.indexing.elasticsearch.EsEmbeddedServer$PluginConfigurableNode.<init>(EsEmbeddedServer.java:72)                                                                                 
        at org.icij.datashare.text.indexing.elasticsearch.EsEmbeddedServer.createNode(EsEmbeddedServer.java:61)                                                                                                    
        at org.icij.datashare.text.indexing.elasticsearch.EsEmbeddedServer.<init>(EsEmbeddedServer.java:37)                                                                                                        
        at org.icij.datashare.mode.EmbeddedMode.configure(EmbeddedMode.java:16)                                                                                                                                    
        at com.google.inject.AbstractModule.configure(AbstractModule.java:62)                                                                                                                                      
        at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)                                                                                                                               
        at com.google.inject.spi.Elements.getElements(Elements.java:110)                                                                                                                                           
        at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)                                                                                                                          
        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)                                                                                                              
        at com.google.inject.Guice.createInjector(Guice.java:99)                                                                                                                                                   
        at com.google.inject.Guice.createInjector(Guice.java:73)                                                                                                                                                   
        at com.google.inject.Guice.createInjector(Guice.java:62)                                                                                                                                                   
        at net.codestory.http.injection.GuiceAdapter.<init>(GuiceAdapter.java:28)                                                                                                                                  
        at org.icij.datashare.mode.CommonMode.defaultRoutes(CommonMode.java:161)                                                                                                                                   
        at org.icij.datashare.mode.CommonMode.lambda$createWebConfiguration$c4402707$1(CommonMode.java:147)                                                                                                        
        at net.codestory.http.routes.RouteCollection.configure(RouteCollection.java:88)                                                                                                                            
        at net.codestory.http.reload.FixedRoutesProvider.<init>(FixedRoutesProvider.java:27)                                                                                                                       
        at net.codestory.http.reload.RoutesProvider.fixed(RoutesProvider.java:29)                                                                                                                                  
        at net.codestory.http.AbstractWebServer.configure(AbstractWebServer.java:62)                                                                                                                               
        at org.icij.datashare.WebApp.lambda$start$0(WebApp.java:31)                                                                                                                                                
        at java.base/java.lang.Thread.run(Thread.java:829)                                                                                                                                                         
Caused by: java.io.IOException: failed to obtain lock on /home/promera/snap/datashare/common/tmp/index/nodes/0                                                                                                     
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:224)                                                                                                                         
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:272)                                                                                                                                  
        ... 22 more                                                                                                                                                                                                
Caused by: java.io.IOException: Mount point not found                                                                                                                                                              
        at java.base/sun.nio.fs.LinuxFileStore.findMountEntry(LinuxFileStore.java:105)                                                                                                                             
        at java.base/sun.nio.fs.UnixFileStore.<init>(UnixFileStore.java:69)                                                                                                                                        
        at java.base/sun.nio.fs.LinuxFileStore.<init>(LinuxFileStore.java:49)                                                                                                                                      
        at java.base/sun.nio.fs.LinuxFileSystemProvider.getFileStore(LinuxFileSystemProvider.java:51)                                                                                                              
        at java.base/sun.nio.fs.LinuxFileSystemProvider.getFileStore(LinuxFileSystemProvider.java:39)                                                                                                              
        at java.base/sun.nio.fs.UnixFileSystemProvider.getFileStore(UnixFileSystemProvider.java:373)                                                                                                               
        at java.base/java.nio.file.Files.getFileStore(Files.java:1488)                                                                                                                                             
        at org.elasticsearch.env.Environment.getFileStore(Environment.java:328)                                                                                                                                    
        at org.elasticsearch.env.NodeEnvironment$NodePath.<init>(NodeEnvironment.java:113)                                                                                                                         
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:218)                                                                                                                         
        ... 23 more  

Any limitation in strict mode that might prevent ElasticSearch from writing to SNAP_USER_COMMON? Or maybe there are additional interfaces that I should use?

For reference, here is my snapcraft.yml:

name: datashare 
base: core20
version: '10.15.3-beta0'
summary: Datashare is a self-hosted search engine for documents.
website: https://datashare.icij.org
description: |
  Datashare is a self-hosted search engine for documents, 
  using Apache Tika and Apache Tesseract to read hundreds
  of file formats. Datashare is developed by the 
  International Consortium of Investigative Journalists (ICIJ), 
  famously known for its groundbreaking investigations
  into the offshore world (Pandora Papers, Panama Papers, etc).
grade: stable
confinement: strict

parts:
  datashare:
    plugin: dump
    source: https://github.com/ICIJ/datashare-installer/releases/download/$SNAPCRAFT_PROJECT_VERSION/datashare-$SNAPCRAFT_PROJECT_VERSION.tgz
    stage-packages:
      - openjdk-11-jre
      - tesseract-ocr
    build-packages:
      - ca-certificates
      - ca-certificates-java
    override-prime: |
      snapcraftctl prime
      chmod +x datashare
      
apps:
  datashare:
    command: datashare
    plugs:
      - home
      - x11
    environment:
      DATASHARE_HOME: $SNAP_USER_COMMON
      DATASHARE_JAR: $SNAP/datashare-dist-$SNAPCRAFT_PROJECT_VERSION-all.jar
      JAVA_HOME: "$SNAP/usr/lib/jvm/java-11-openjdk-amd64"
      JAVA_BIN: "$JAVA_HOME/bin/java"
      PATH: "$SNAP/bin:$JAVA_HOME/bin:$PATH"

Thanks!

I built the snap locally and the lock for node 0 is created in $SNAP_USER_COMMON. It’s also trying to create network sockets and perform other system-related operations, so you can try to add network, network-bind, system-observe, and mount-observe to the plugs list. Note that mount-observe and system-observe don’t auto-connect, you need to connect them manually with snap connect datashare:mount-observe and snap connect datashare:system-observe (or request auto-connect permissions for those interfaces).

Here’s what I get after adding and connecting the missing plugs:

...
2022-12-26 14:37:11,018 [Thread-0] INFO  ExtensionLoader - read directory /home/claudio/snap/datashare/common/extensions and found jars (executable): []
2022-12-26 14:37:11,034 [Thread-0] INFO  HikariDataSource - HikariPool-1 - Starting...
2022-12-26 14:37:11,106 [Thread-0] INFO  HikariDataSource - HikariPool-1 - Start completed.
2022-12-26 14:37:11,157 [Thread-0] INFO  HikariDataSource - HikariPool-2 - Starting...
2022-12-26 14:37:11,157 [Thread-0] INFO  HikariDataSource - HikariPool-2 - Start completed.
2022-12-26 14:37:11,160 [Thread-0] INFO  HikariDataSource - HikariPool-3 - Starting...
2022-12-26 14:37:11,161 [Thread-0] INFO  HikariDataSource - HikariPool-3 - Start completed.
2022-12-26 14:37:11,179 [Thread-0] INFO  HikariDataSource - HikariPool-4 - Starting...
2022-12-26 14:37:11,180 [Thread-0] INFO  HikariDataSource - HikariPool-4 - Start completed.
2022-12-26 14:37:14,621 [Thread-0] INFO  JdbcExecutor - SELECT COUNT(*) FROM DATABASECHANGELOGLOCK
2022-12-26 14:37:14,625 [Thread-0] INFO  JdbcExecutor - SELECT COUNT(*) FROM DATABASECHANGELOGLOCK
2022-12-26 14:37:14,627 [Thread-0] INFO  JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2022-12-26 14:37:14,639 [Thread-0] INFO  StandardLockService - Successfully acquired change log lock
2022-12-26 14:37:15,655 [Thread-0] INFO  JdbcExecutor - SELECT MD5SUM FROM DATABASECHANGELOG WHERE MD5SUM IS NOT NULL
2022-12-26 14:37:15,656 [Thread-0] INFO  JdbcExecutor - SELECT COUNT(*) FROM DATABASECHANGELOG
2022-12-26 14:37:15,656 [Thread-0] INFO  StandardChangeLogHistoryService - Reading from DATABASECHANGELOG
2022-12-26 14:37:15,656 [Thread-0] INFO  JdbcExecutor - SELECT * FROM DATABASECHANGELOG ORDER BY DATEEXECUTED ASC, ORDEREXECUTED ASC
2022-12-26 14:37:15,701 [Thread-0] INFO  StandardLockService - Successfully released change log lock
2022-12-26 14:37:15,702 [Thread-0] INFO  HikariDataSource - HikariPool-4 - Shutdown initiated...
2022-12-26 14:37:15,705 [Thread-0] INFO  HikariDataSource - HikariPool-4 - Shutdown completed.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/snap/datashare/x7/datashare-dist-10.15.3-beta0-all.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2022-12-26 14:37:15,854 [Thread-0] INFO  Indexer - indexer defined with cfg{indexJoinField='join', docTypeField='type', shards=1, replicas=1}
2022-12-26 14:37:15,880 [Thread-0] INFO  ExtensionLoader - read directory /home/claudio/snap/datashare/common/extensions and found jars (executable): []
2022-12-26 14:37:15,960 [pool-4-thread-1] INFO  IndexWaiterFilter - Ping elasticsearch succeeded
2022-12-26 14:37:16,204 [Thread-0] INFO  Fluent - Production mode
2022-12-26 14:37:16,213 [Thread-0] INFO  Fluent - Server started on port 8080
1 Like

Thanks @cmatsuoka! Worked like a charm!

For the record, I just had to tweak the JNA tmpdir option to write into $SNAP_USER_COMMON:

My last problem is the $HOME variable not being set to the correct value, even with the home plug.

EDIT: obviously we can use /home/$USER to infer the homedir but I would love to understand why the $HOME is not correct out of the box.