F-string formatting syntaxError

m-yahya · February 12, 2020, 6:59pm

Hi all,
I’m trying to create a snap for my python application.
The snap is created successfully. But when I try to run the app using the command, I’m getting syntax error because of f-string formatting. An example of error line is:
prefix = f’{project_id.upper()}/OLI_{oli_box_id}/{device_type.upper()}’
^
SyntaxError: invalid syntax

any help how can I use f-string formatting in modules so that I can create snap without error?
thanks

kyrofa · February 12, 2020, 7:02pm

Are you specifying a base in your snapcraft.yaml? f-strings are only available as of Python 3.6, but Xenial (i.e. base: core) is only 3.5. Bionic (base: core18) has 3.6.

m-yahya · February 12, 2020, 7:04pm

I’m using base: core18 in .yaml file.
my operating system is Ubuntu 19.10

m-yahya · February 12, 2020, 7:22pm

@kyrofa should I downgrade to Bionic?
or is there any other solution?
thanks

kyrofa · February 12, 2020, 7:43pm

Any chance you can share your project, or provide a minimal reproducer?

m-yahya · February 12, 2020, 7:59pm

@kyrofa can you please look at this repo:

inside the directory, I’m doing the following steps:
$ snapcraft --debug
$ sudo snap install olibox-core_0.1dev_amd64.snap --devmode --dangerous

run the app with following command:
$ olibox-core

m-yahya · February 13, 2020, 9:28am

@kyrofa from the official documentation I can see there are only two supported bases:

core18 based on Ubuntu 18.04 LTS
core based on Ubuntu 16.04 LTS
also from the documentation:

If there are specific dependencies that cannot be easily met with core18 then core is a valid and supported alternative.

so I tried the core and without any base as well but no luck.
any help please?
thanks

jamesh · February 13, 2020, 10:18am

It looks like that f-string depends on variables imported from olibox_core.olibox_core.olibox_pkg.environments. In turn, that module seems to load its globals from a TOML file:

github.com

m-yahya/test-snap/blob/ff82e03634bc9f4000d9721edf5ed93aff82e98c/olibox_core/olibox_core/olibox_pkg/environments.py#L5


import os
import toml
if os.path.isfile('config.toml'):
    f = toml.load('config.toml')
    items = f.keys()
    unique_dict = {}
    for item in items:
        if type(f[item]) is not str:
            for i in f[item].keys():
                unique_dict[i] = f[item][i]
        else:
            unique_dict[item] = f[item]

If it can’t find a config.toml file in the current directory (such as the one in the root of your repository), you won’t have the project_id, oli_box_id, and device_type variables, producing the syntax error you got.

m-yahya · February 13, 2020, 10:28am

@jamesh thank you very much for your help.
Yes you are right, olibox_core.olibox_core.olibox_pkg.environments is loading variables for config.toml file that is located is the root directory.
I’m using setup.py for that I have to specify the path relative to setup.py. if I install the app using pip install . and run the command inside the directory, it works fine.

I just changed the path of config.toml file relative to the environments.py module (../../config.toml9) and remove and rebuild the snap, but still getting the same error.

m-yahya · February 13, 2020, 10:31am

I wonder the error is still same even I changed the path:

File "/snap/olibox-core/x1/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/config.py", line 104
    """
      ^
SyntaxError: invalid syntax

it is still pointing to the config.toml file line 104 that has closing quotes.
I think file is detected.

jamesh · February 13, 2020, 10:38am

You are not going to have access to the project source directory from your snap, so you will need to ensure it is installed, and that your app looks for it in an appropriate location.

Your app will be run with a number of environment variables that you can use to locate files. $SNAP will be set to the directory holding the (read-only) contents of your snap. If you intend for the user to modify the file, you might instead want to look for it in $SNAP_DATA or $SNAP_USER_DATA (depending on whether this is intended to be a system wide service or not).

If you are using $SNAP_DATA, you will want to do one of two things:

have your app fall back to a default configuration file in $SNAP if one in $SNAP_DATA is not found.
add an install hook to your snap that copies a default configuration to $SNAP_DATA.

m-yahya · February 13, 2020, 10:46am

so you will need to ensure it is installed

do I need to install the app using pip install .?

I’m at the very early stage of snap building. If I run the app it will first the user for multiple inputs and 'll save those user inputs to the config.toml file. The app then read out variables form toml file and uses for some other functionality. can you please help me out a bit more how to proceed now?
many thanks

jamesh · February 13, 2020, 10:50am

You won’t be able to write to any locations under /snap/olibox-core – it is all read only (your app is mounted as a squashfs file system). You will need to modify it so it can search for or store its configuration some place else. The environment variables I mentioned previously are probably what you want.

m-yahya · February 13, 2020, 11:05am

Currently, I can only see .yaml the file inside the snap directory even after creating the snap.
is this normal behavior or some other files should also be generated?

is that possible for you to provide a link to some resources or minimum example of storing user inputs and then using those configs later inside the snap?

jamesh · February 13, 2020, 11:27am

When you built your snap, you should have ended up with a package with the file extension .snap. You can see the contents of the package by running unsquashfs -l filename.snap. This is what you will see at /snap/olibox-core/current when you install it. Nothing else in your repository will be available on the user’s system.

m-yahya · February 13, 2020, 11:29am

I want to make things a bit simple. as config file is already there (containing all the variables), I’m wondering why it is not readable?

byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/environments.py to environments.cpython-35.pyc
byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/write_json.py to write_json.cpython-35.pyc
  File "/root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/write_json.py", line 12
    if os.path.isfile(f'../../tmp/{file}'):
                                        ^
SyntaxError: invalid syntax

byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/oli_ir.py to oli_ir.cpython-35.pyc
  File "/root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/oli_ir.py", line 45
    print(f'Power load is {power_load}')
                                      ^
SyntaxError: invalid syntax

byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/oli_restapi.py to oli_restapi.cpython-35.pyc
  File "/root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/oli_restapi.py", line 17
    url = f'http://{oli_restapi_ip}:{oli_restapi_port}{oli_restapi_query}'
                                                                         ^
SyntaxError: invalid syntax

byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/oli_mqtt.py to oli_mqtt.cpython-35.pyc
  File "/root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/oli_mqtt.py", line 51
    client_id = f'OLI_{oli_box_id}_PUB'
                                      ^
SyntaxError: invalid syntax

byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/modbus_config.py to modbus_config.cpython-35.pyc
byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/smartcontract_config.py to smartcontract_config.cpython-35.pyc
byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/__init__.py to __init__.cpython-35.pyc
byte-compiling /root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/helpers.py to helpers.cpython-35.pyc
  File "/root/parts/olibox-core/install/lib/python3.5/site-packages/olibox_core/olibox_core/olibox_pkg/helpers.py", line 78
    prefix = f'{project_id.upper()}/OLI_{oli_box_id}/{device_type.upper()}'
                                                                          ^
SyntaxError: invalid syntax

jamesh · February 13, 2020, 11:40am

Judging by those errors, you’re trying to build against Python 3.5 again, which does not support f-strings. Are you still building against core18?

It also looks like a lot of your code is searching for configuration files relative to the current working directory still. So again: I suggest using the environment variables snapd makes available to you to decide where to put your configuration files.

m-yahya · February 13, 2020, 11:43am

yes, I’m using base: core18. This is the .yaml file:

name: olibox-core
base: core18
version: '0.1dev'
summary: Core functionality for Oli box
description: |
    Olibox_core module provides the basic functionality 
    to wrtie data from IR sensor and REST API to json files
    and publish this data to mqtt broker.
    This utility is particularly useful for Oli box.
base: core
grade: devel
confinement: devmode

parts:
  olibox-core:
    source: https://github.com/olisystems/olibox_core.git
    source-tag: 0.1dev
    plugin: python
    source: .
    requirements: ['requirements.txt']

apps:
  olibox-core:
    command: bin/olibox-core

do I need to move config.toml file to some directory?
or do I need to make some changes to .yaml file to achieve this?

m-yahya · February 13, 2020, 11:46am

I’m using following versions:


Python 3.7.4

snap    2.43.2
snapd   2.43.2
series  16
ubuntu  19.10
kernel  5.3.0-29-generic

snapcraft, version 3.9.8

os ubuntu 19.10

ogra · February 13, 2020, 12:24pm

what is the exact line you use to run a build (are you somehow preventing multipass builds ?)