How to persist user config data to SNAP_USER_DATA?

m-yahya · February 13, 2020, 5:13pm

I have created a snap of my application using snapcraft --debug.

The application at its core first shows two prompt asking user to enter name and age. The user input data is then saved into config.toml file in the root directory. Next, the application will print out the user entered data.

The current behavior of the snap is it asks the user to enter the data and prints the data that was previously entered. This means the data entered now will be printed when the application will run next time.

I got the idea of using SNAP_USER_DATA to save the user config but I’m not sure about that how can I effectively use it.

I wanted to persist data of config file to SNAP_USER_DATA.
To break down the question, I wanted to perform the following steps:

create a config file inside SNAP_USER_DATA. From the stack overflow, I can do this by:

$ mkdir -p snap/hooks
$ echo "touch \$SNAP_USER_DATA/config.toml" >> snap/hooks/install
$ chmod a+x snap/hooks/install

is that possible to specify this is .yaml file? so that when I create the snap this file should also be created at run time.

next either I directly want to save the user input into $SNAP_USER_DATA/config.toml or copy from config.toml to $SNAP_USER_DATA/config.toml.
finally, I want to use config from $SNAP_USER_DATA/config.toml inside my app.

here is the link of my app: https://github.com/m-yahya/snap-tutorial
any help will be much appreciated.
thanks

ogra · February 14, 2020, 10:13am

m-yahya:

create a config file inside SNAP_USER_DATA. From the stack overflow, I can do this by:
$ mkdir -p snap/hooks
$ echo "touch \$SNAP_USER_DATA/config.toml" >> snap/hooks/install
$ chmod a+x snap/hooks/install

the install hook runs as root so this would just install the config into SNAP_USER_DATA in roots homedir (/root) …

if you really need to pre-create the file (does the app not work if it is not existing ?) i’d suggest using a wrapper script like:

#! /bin/sh

[ -e "$SNAP_USER_DATA/config.toml" ] || touch $SNAP_USER_DATA/config.toml

exec "$@"

ship it in bin/ inside your snap and change your command: entry:

from:

apps:
  myapp:
    command: myapp

to:

apps:
  myapp:
    command: mylauncher.sh myapp

(or you could use a command chain)

well, then just make your app read and use the environment variable when handling the config file …

m-yahya · February 14, 2020, 10:18am

app will not work if this file is not present or empty.
my application is designed is such a way that when I run the application it 'll first ask user to enter couple of inputs. The app will save these inputs to config.toml. The rest of the app functionality will read through the config.toml and use variables from there.

m-yahya · February 14, 2020, 10:24am

do I need to create snap/bin/mylauncher.sh and put the contents inside .sh file?

m-yahya · February 14, 2020, 10:38am

I’m getting the following warning, is there any way to fix/silent this warning:

The 'snap' directory is meant specifically for snapcraft, but it contains the following non-snapcraft-related paths, which is unsupported and will cause unexpected behavior:
- bin
- bin/mylauncher.sh

If you must store these files within the 'snap' directory, move them to 'snap/local', which is ignored by snapcraft.

ogra · February 14, 2020, 11:07am

try something like (if mylauncher.sh is in the toplevel dir of your source, else adjust the paths of cp accordingly):

parts:
  ...
  launcher:
    plugin: nil
    source: .
    override-build: |
      mkdir -p $SNAPCRAFT_PART_INSTALL/bin
      cp -av mylauncher.sh $SNAPCRAFT_PART_INSTALL/bin/

(and make sure mylauncher.sh is executable)

also note: the snap/ dir is reserved for snap specidic metadata, while there is a way to override that i would keep the scripts separate from the packaging metadata.

m-yahya · February 14, 2020, 11:19am

The snap is now created.
If I run the command olibox-core, it throws the error:

PermissionError: [Errno 13] Permission denied: 'config.toml'.

do I need to run any other command before?
olibox-core.mypp is not working.

ogra · February 14, 2020, 11:37am

github.com

m-yahya/snap-tutorial/blob/master/olibox_core/olibox_core/olibox_pkg/config.py#L25


 # This is a OLI Box TOML config file.
 title = "OLI BOX Config"
 [user_data]
 name = '{name}'
 age = '{age}'
 
 """
parsed_config = toml.loads(config)
formatted_config = toml.dumps(parsed_config)
with open(snap_userdata + '/config.toml', 'w') as f:
    f.write(formatted_config)

    with open('config.toml', 'w') as f:
        f.write(formatted_config)

with that code it tries to write the file to ./ … if the app is executed, but you want to write to SNAP_USER_DATA:

    snap_userdata = os.environ['SNAP_USER_DATA']

    with open(snap_userdata + '/config.toml', 'w') as f:
        f.write(formatted_config)

(and change the rest of your code whereever you handle this file in a similar way (probably with a few more tests etc))

BTW: the systemd journal should have “DENIED” messages that point out the full path that was attempted to write to …

m-yahya · February 14, 2020, 11:54am

I have updated the file path like:

import os

import toml

snap_userdata = os.environ['SNAP_USER_DATA']

if os.path.isfile(snap_userdata + 'config.toml'):
    f = toml.load(snap_userdata + 'config.toml')

    items = f.keys()
    unique_dict = {}
    for item in items:
        if type(f[item]) is not str:
            for i in f[item].keys():
                unique_dict[i] = f[item][i]
        else:
            unique_dict[item] = f[item]

    locals().update(unique_dict)

    # for key, val in new_dict.items():
    #     exec(key + '=val')

else:
    print('file not found')

when I run the app it says:

file not found
enter your name:

and after entering the input, it throws the error:

file not found
enter your name: john doe
enter your age: 25
Traceback (most recent call last):
  File "/snap/olibox-core/x1/bin/olibox-core", line 11, in <module>
    load_entry_point('olibox-core==0.1.dev0', 'console_scripts', 'olibox-core')()
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/olibox_core/olibox_core/olibox_core.py", line 11, in init
    pkg.init()
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/olibox_core/olibox_core/olibox_pkg/config.py", line 27, in init
    with open(snap_userdata + 'config.toml', 'w') as f:
PermissionError: [Errno 13] Permission denied: '/home/yahya/snap/olibox-core/x1config.toml'
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/olibox_core/olibox_core/olibox_core.py", line 22, in main
    print_user()
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/olibox_core/olibox_core/olibox_core.py", line 16, in print_user
    user = pkg.get_user()
  File "/snap/olibox-core/x1/lib/python3.6/site-packages/olibox_core/olibox_core/olibox_pkg/user.py", line 12, in get_user
    print(f'this is working{name}')
NameError: name 'name' is not defined

ogra · February 14, 2020, 12:01pm

m-yahya:

return callback(*args, **kwargs)
File “/snap/olibox-core/x1/lib/python3.6/site-packages/olibox_core/olibox_core/olibox_pkg/config.py”, line 27, in init
with open(snap_userdata + ‘config.toml’, ‘w’) as f:
PermissionError: [Errno 13] Permission denied: ‘/home/yahya/snap/olibox-core/x1config.toml’

the path doesnt end with a slash …

if os.path.isfile(snap_userdata + '/config.toml'): 
    f = toml.load(snap_userdata + '/config.toml')

m-yahya · February 14, 2020, 12:01pm

After installing the snap, I’m just calling the olibox-core command.
I’m not sure if I need to run anyother command as well. For example to run the script

ogra · February 14, 2020, 12:08pm

running the script is covered though the command: entry, i just messed up the path in my suggestion above with a missing slash, just add it everywhere …

(fixed the suggestion above too for any bystanders reading this later)

m-yahya · February 14, 2020, 12:56pm

The path error is fixed.
But I always get the last input, not the current. For example:

$ olibox-core
enter your name: doe
enter your age: 30
this is workingfirst name
This is test print of This is test var
this is ('first name', '20')

in the above snipper, ‘doe’ and ‘30’ will be printed out on the next run.

ogra · February 14, 2020, 1:23pm

well, that must be soemthing inside your app, surely not related to the snapping

m-yahya · February 14, 2020, 1:40pm

In my app, I’m calling the init function that asks user for the input and write inputs to config file.
after the execution of this function (at exit), I’m calling print config function.

after creating the snap, when I run the app first time I can see file not found that is being called from the print function. but if I run the snap again then it print function executes after the init. so only first run is not in order.

I’m not sure why it is being called before the init function(in my app, not snap, it was executing after the init).

m-yahya · February 14, 2020, 3:00pm

after installing the snap when I run the application, it creates olibox-core directory inside ~/snap.
at this point there is no config file in current directory. I get the file not found.

after running the app 2nd time, config file is created in the current directory but still file not found issue.

3rd time app works as expected but prints previously entered data.

ogra · February 14, 2020, 4:08pm

well, that you do not run the launcher at all when running olibox-core might be an issue

https://github.com/m-yahya/snap-tutorial/blob/master/snap/snapcraft.yaml#L27

when i talked about myapp above i indeed meant your app … i.e. do not add a myapp: entry but modify the existing one like:

apps:
  olibox-core: 
    command: mylauncher.sh olibox-core
....

m-yahya · February 14, 2020, 4:22pm

I’m so sorry, I thought may be I need a separate command to run the launcher.
I removed the extra command to:

apps:
  olibox-core:
    command: mylauncher.sh olibox-core

and I’m running the snap using the command olibox-core. There is no file not found issue. But I’m still getting the previous input printing, not the current one:weary:

m-yahya · February 14, 2020, 4:42pm

I tested it again. The order of steps is:

First read out the config file
ask user to enter the input
print the data read out in step 1
save the user input to config file.

I think this is because the function that prints data from config file is in user module inside the olibox_pkg. The olibox_pkg imports the print_user function to make it available when package is imported into other module. When the package is imported inside code module, the print_user function reads out the config file and prints out later (after the init function).
that’s why it always prints the previous data.

I’m not sure if I understood it correct, any help please how to fix it?
many thanks

m-yahya · February 17, 2020, 10:14am

If I want to create to more files in $SNAP_USER_DATA with some initial contents (for example I wanted to have ‘{}’ in my files), is this a valid script for that:

#! /bin/sh

[ -e "$SNAP_USER_DATA/config.toml" ] || touch $SNAP_USER_DATA/config.toml

[ -e "{}" > "$SNAP_USER_DATA/energy-data.json" ] || touch $SNAP_USER_DATA/energy-data.json

[ -e "{}" > "$SNAP_USER_DATA/power-data.json" ] || touch $SNAP_USER_DATA/power-data.json

exec "$@"

and then how can I access the individual files?
Before I’m using snap_userdata = os.environ['SNAP_USER_DATA'] but I’m not sure how to specify a file if there are more than one files.
is this correct snap_userdata + '/energy-data.json'?
many thanks