Using external metadata

sergiusens · March 24, 2018, 6:03pm

To help avoid unnecessary duplication, and for convenience, Snapcraft can process and incorporate external metadata from within snapcraft.yaml by using parse-info within a part and a corresponding adopt-info key.

For example, the following snapcraft.yaml will parse a file called metadata-file. Snapcraft will attempt to extract version, summary and description metadata for the snap, all of which are mandatory:

name: my-snap-name
adopt-info: part-with-metadata

parts:
  part-with-metadata:
    plugin: dump
    source: .
    parse-info: [metadata-file]

See The snapcraft format for further details on Snapcraft metadata and how it’s used.

Source types

An external metadata source can be one of the following:

AppStream: a standard for software components
Scriptlets: a snapcraftctl-driven command to generate version and grade

See below for details on incorporating each of the above into your snapcraft.yaml.

AppStream^⚓

AppStream is a metadata standard used to describe a common set software components. It can be parsed by snapcraft to provide the title, version, summary, description and icon for a snap, along with the location of an app’s desktop file.

The following is a typical example from an upstream project. It’s an AppStream file called sampleapp.metainfo.xml:

<?xml version="1.0" encoding="UTF-8"?>
<component type="desktop-application">
  <id>com.example.sampleapp</id> 
  <name>Sample App</name>
  <summary>Single-line elevator pitch for your amazing application</summary>
  <description>
    This is applications's description. A paragraph or two to tell the
    most important story about it.
  </description>
  <icon type="local">assets/icon.png</icon>
  <launchable type="desktop-id">
    com.example.sampleapp.desktop
  </launchable>
  <releases>
    <release date="2019-11-27" version="4.2.8.0"/>
  </releases>
</component>

We adopt the above metadata into snapcraft.yaml with the following:

name: sampleapp-name
adopt-info: sampleapp

apps:
  sampleapp:
    command: sampleapp
    common-id: com.example.sampleapp

parts:
  sampleapp:
    plugin: dump
    source: http://github.com/example/sampleapp.git
    parse-info: [usr/share/metainfo/com.example.sampleapp.appdata.xml]

ⓘ The path in parse-info is a relative path from the part source, build or install directory (SNAPCRAFT_PART_SRC, SNAPCRAFT_PART_BUILD, SNAPCRAFT_PART_INSTALL).

The resulting snap will use the title, version, summary and description from the AppStream file.

You can also link each app in your snap to specific AppStream metadata by pointing the common-id key of that app to the component id field in the AppStream metadata. Snapcraft will use the metadata of that component to get the .desktop entry file for that app.

ⓘ For backwards compatibility, some component ids in the AppStream metadata have a .desktop suffix. If this is the case for your application, the common-id of your app should also use that suffix.

Note: The process to get the .desktop file entry from the AppStream metadata goes as follows. First, Snapcraft searches for a parsed AppStream file with the same component id as the app’s common-id and extracts the Desktop File ID (desktop-id) from that component. If that component doesn’t specify a desktop-id, Snapcraft will use the component id as the Desktop File ID. Snapcraft will then search for a desktop file matching the Desktop File ID in the usr/local/share and usr/share directories relative to the part source, and by following the Desktop File ID rules.

Part scriptlets^⚓

Individual parts in your snapcraft.yaml can set the version and grade by using snapcraftctl. All you need to do is select which part to adopt using adopt-info:

# ...
adopt-info: my-part
# ...
parts:
  my-part:
    # ...
    override-pull: |
      snapcraftctl pull
      snapcraftctl set-version "my-version"
      snapcraftctl set-grade "devel"

See Scriptlets for more details on using scripting elements within snapcraft.yaml.

[not recommended] `setup.py`^⚓

Using parse-info with setup.py is currently discouraged because it has many issues. For example, it incorrectly uses the project’s summary as the snap’s description and it might crash the snap build.

A setup.py file is used by many Python projects to help with package installation. If your setup.py uses setuptools and defines version and description, these can be extracted from setup.py and used as the version and description metadata in the resulting snap.

The following is an example setup.py in the root of a hypothetical git tree:

import setuptools

setuptools.setup(
    name='hello-world',
    version='1.0',
    author='snapcrafter',
    author_email='snapcraft@lists.snapcraft.io',
    description='A simple hello world in python',
    scripts=['hello']
)

You can adopt the relevant metadata in the above with the following snapcraft.yaml

name: sampleapp-name
summary: sampleapp summary
adopt-info: sampleapp

apps:
  sampleapp:
    command: sampleapp

parts:
  sampleapp:
    plugin: python
    source: http://github.com/example/sampleapp.git
    parse-info: [setup.py]

Snapcraft versions and compatibility^⚓

Change	snapcraft version
Initial introduction	`2.39`
appstream support	`2.39`
`common-id`	`2.40`
`setup.py` support	`2.41`
snapcraftctl set-version	`2.41`
snapcraftctl set-grade	`2.41`

lucyllewy · March 24, 2018, 9:22pm

What happens if you include multiple items in the list in a part’s parse-info?
Is there a usecase for parse-info that isn’t linked to the snap using adopt-info?
- if not then why do we have adopt-info at all?
Can we adopt the info from multiple parts?
- if we can have multiple items in parse-info for a single part then surely we should be able to reference the info from multiple parts?

sergiusens · March 26, 2018, 4:47pm

I will reflect the docs with the behavior of in function precedence.

You could be using someone else’s part that suddenly starts using parse-info. This provides a very explicit behavior from day one. Do you think we should relax it? This documentation mostly reflects Extracting existing data from projects to feed into snap.yaml - #8 by niemeyer

I think you should move to using the equivalents of the to-be set-version and such if you have a very odd use case or we will be creating a Swiss knife of a beast

lucyllewy · March 26, 2018, 7:03pm

Thanks for the clarifications That all sounds rather tasty nomnomnom…

No, I think that makes sense now that you’ve blown the cobwebs from my brain :-p

Lin-Buo-Ren · November 1, 2018, 6:49am

I wonder if it’s possible to adopt the git describe version from the adopt-info specified part’s source repo when the version key is also set with the value git

pedronis · November 6, 2018, 12:41pm

I think something like that is available or in the work, cc @sergiusens

sergiusens · November 8, 2018, 9:05am

Can you clarify? In the meantime I will explain what is already there.

version: git

Works, it is just very opinionated on release tags, from the git man page:

Annotated tags are meant for release while lightweight tags are meant
for private or temporary object labels. For this reason, some git
commands for naming objects (like git describe) will ignore lightweight
tags by default.

For this and other reasons we now have snapcraftctl set-version <version> which can be used in any part and can be picked up by àdopt-info`.

Lin-Buo-Ren · November 11, 2018, 4:26pm

I would like to use the version: git like version string, but instead of describing the recipe containing repo, make it describe the part’s source repo according to the adopt-info key’s value. The described version is enough for me and I want to avoid calling snapcraftctl set-version <version> when possible.

Lin-Buo-Ren · November 11, 2018, 4:29pm

I’ve encountered a problem that in snapcraft 2.43.1(snap) the snapcraftctl command can’t be called in a script called in a override-pull scriptlet:

Pulling hello 
Cloning into '/root/build_hello-snapcrafters/parts/hello/src'...

...

/root/build_hello-snapcrafters/parts/hello/src/../../../snap/local/utilities/set-snap-version.bash: line 35: snapcraftctl: command not found
Failed to run 'override-pull': Exit code was 127.

This issue isn’t reproduced in build.snapcraft.io, nor it is reproduced in snapcraft 3.0

Lin-Buo-Ren · January 25, 2019, 9:20pm

According to the setuptools documentation the description key is equivalent to Snapcraft’s summary key and the long_description key is equivalent to Snapcraft’s description key, however currently Snapcraft uses the description key’s value as the snap description and ignores the long_description key in setup.py, I would like to request changing the behavior to match with setuptools.

UPDATE: Bug #1813364 “The setup.py parse-info method treats description/long_description keys wrongly” : Bugs : Snapcraft

sergiusens · January 28, 2019, 12:48pm

Hi, good catch, if you would want to work on this it would be welcomed!

Lin-Buo-Ren · January 30, 2019, 7:14pm

I’d like to, but unfortunately I’m currently not technically capable of doing so as I don’t speak either Python or Go…

sergiusens · January 31, 2019, 8:57pm

@degville hey, I asked this by other means, it may have been missed due to that, the ask is, can we get this on the main docs somehow? When I asked, it wasn’t even searchable (using the same query which yields results here on the forum: “adopt-info”)

degville · January 31, 2019, 9:19pm

…sorry, this must have fallen off my radar. But yes, definitely. I’ll get it added and make sure it’s returned in the search results. Thanks for the prod!

alan_g · February 1, 2019, 10:45am

It looks to me as though there’s a missing metadata source: deb package information.

It isn’t hard to parse the output from, say, apt-cache policy and use snapcraftctl but I can’t be the only one doing that?

ijohnson · March 27, 2019, 7:44pm

I took a stab at a PR for this here: https://github.com/snapcore/snapcraft/pull/2518

dhoomakethu · June 14, 2019, 5:12am

I am trying to use scriptlets to set the version for my snap. The build fails every once in a while with Failed to generate snap metadata: 'adopt-info' refers to part 'my-part', but that part is lacking the 'parse-info' property. . I am running snapcraft v3.5 and building on Ubuntu 16.04 with my base set to core. If I clean only the part (python) that is causing this failure and re run snapcraft, the build goes fine and a snap is generated.

Here is my override-pull

override-pull: |
      snapcraftctl pull
      snapcraftctl set-version "$(python3 -c 'from mypackage import version; print(version.__version__)')"

I am referring https://docs.snapcraft.io/using-external-metadata#meta-scriptlet .

Any Idea why is this happening ?

sergiusens · June 15, 2019, 7:55pm

Does this only happen when “re-running” or on first run?

dhoomakethu · June 15, 2019, 11:20pm

Happens on the first run or any clean run, If I clean the part and build again the build goes through.

jamesh · July 9, 2019, 8:39am

One thing missing from the description of the parse-info key is a description of how the metadata file is located.

Is it expected to be in the part’s src directory? The build directory? Or the install directory? If it searches multiple locations, which takes precedence?
At which step of the build process is the metadata extracted?

Using external metadata

Source types

AppStream⚓

Part scriptlets⚓

[not recommended] `setup.py`⚓

Snapcraft versions and compatibility⚓

AppStream^⚓

Part scriptlets^⚓

[not recommended] `setup.py`^⚓

Snapcraft versions and compatibility^⚓