Using external metadata

sergiusens · 2018-03-24 18:03:10 UTC

To help avoid unnecessary duplication, and for convenience, Snapcraft can process and incorporate external metadata from within snapcraft.yaml by using parse-info within a part and a corresponding adopt-info key.

For example, the following snapcraft.yaml will parse a file called metadata-file. Snapcraft will attempt to extract version, summary and description metadata for the snap, all of which are mandatory:

name: my-snap-name
adopt-info: part-with-metadata

parts:
  part-with-metadata:
    plugin: dump
    source: .
    parse-info: [metadata-file]

See The snapcraft format for further details on Snapcraft metadata and how it’s used.

Source types

An external metadata source can be one of the following:

Appstream: a standard for software components
setup.py: commonly used by Python projects to help with package installation
Scriptlets: a snapcraftctl-driven command to generate version and grade

See below for details on incorporating each of the above into your snapcraft.yaml.

Appstream

Appstream is a metadata standard used to describe a common set software components. It can be parsed by snapcraft to provide the summary, description and icon for a snap, along with the location of an app’s desktop file.

The following is a typical example from an upstream project. It’s an appstream file called sampleapp.metainfo.xml:

<?xml version="1.0" encoding="UTF-8"?>
<component type="desktop-application">
  <id>com.example.sampleapp</id>
  <summary>Single-line elevator pitch for your amazing application</summary>
  <description>
    This is applications's description. A paragraph or two to tell the
    most important story about it.
  </description>
  <icon type="local">assets/icon.png</icon>
  <launchable type="desktop-id">
    com.example.sampleapp.desktop
  </launchable>
</component>

We adopt the above metadata into snapcraft.yaml with the following:

name: sampleapp-name
adopt-info: sampleapp

apps:
  sampleapp:
    command: sampleapp
    common-id: com.example.sampleapp

parts:
  sampleapp:
    plugin: dump
    source: http://github.com/example/sampleapp.git
    parse-info: [sampleapp.metainfo.xml]

The resulting snap will take the summary and description from the appstream file as well as using the referred icon and desktop files for the app.

Appstream uses a Desktop File ID instead of a path to declare a desktop file.

To link a desktop-id within the appstream file to your app, you must define a common-id in snapcraft.yaml.

Snapcraft will search for a parsed appstream file with the same component identifier, com.example.sampleapp in the above example, and extract the desktop-id from there. It will then search the usr/local/share and usr/share directories relative to the part source, and by following the Desktop File ID rules.

`setup.py`

A setup.py file is used by many Python projects to help with package installation. If your setup.py uses setuptools and defines version and description, these can be extracted from setup.py and used as the version and description metadata in the resulting snap.

The following is an example setup.py in the root of a hypothetical git tree:

import setuptools

setuptools.setup(
    name='hello-world',
    version='1.0',
    author='snapcrafter',
    author_email='snapcraft@lists.snapcraft.io',
    description='A simple hello world in python',
    scripts=['hello']
)

You can adopt the relevant metadata in the above with the following snapcraft.yaml

name: sampleapp-name
summary: sampleapp summary
adopt-info: sampleapp

apps:
  sampleapp:
    command: sampleapp

parts:
  sampleapp:
    plugin: python
    source: http://github.com/example/sampleapp.git
    parse-info: [setup.py]

Part scriptlets

Individual parts in your snapcraft.yaml can set the version and grade by using snapcraftctl. All you need to do is select which part to adopt using adopt-info:

# ...
adopt-info: my-part
# ...
parts:
  my-part:
    # ...
    override-pull: |
      snapcraftctl pull
      snapcraftctl set-version "my-version"
      snapcraftctl set-grade "devel"

See Scriptlets for more details on using scripting elements within snapcraft.yaml.

Snapcraft versions and compatibility

Change	snapcraft version
Initial introduction	`2.39`
appstream support	`2.39`
`common-id`	`2.40`
`setup.py` support	`2.41`
snapcraftctl set-version	`2.41`
snapcraftctl set-grade	`2.41`

daniel · 2018-03-24 21:22:47 UTC

What happens if you include multiple items in the list in a part’s parse-info?
Is there a usecase for parse-info that isn’t linked to the snap using adopt-info?
- if not then why do we have adopt-info at all?
Can we adopt the info from multiple parts?
- if we can have multiple items in parse-info for a single part then surely we should be able to reference the info from multiple parts?

sergiusens · 2018-03-26 16:47:50 UTC

I will reflect the docs with the behavior of in function precedence.

You could be using someone else’s part that suddenly starts using parse-info. This provides a very explicit behavior from day one. Do you think we should relax it? This documentation mostly reflects Extracting existing data from projects to feed into snap.yaml

I think you should move to using the equivalents of the to-be set-version and such if you have a very odd use case or we will be creating a Swiss knife of a beast

daniel · 2018-03-26 19:03:01 UTC

Thanks for the clarifications That all sounds rather tasty nomnomnom…

No, I think that makes sense now that you’ve blown the cobwebs from my brain :-p

Lin-Buo-Ren · 2018-11-01 06:49:15 UTC

I wonder if it’s possible to adopt the git describe version from the adopt-info specified part’s source repo when the version key is also set with the value git

pedronis · 2018-11-06 12:41:55 UTC

I think something like that is available or in the work, cc @sergiusens

sergiusens · 2018-11-08 09:05:54 UTC

Can you clarify? In the meantime I will explain what is already there.

version: git

Works, it is just very opinionated on release tags, from the git man page:

Annotated tags are meant for release while lightweight tags are meant
for private or temporary object labels. For this reason, some git
commands for naming objects (like git describe) will ignore lightweight
tags by default.

For this and other reasons we now have snapcraftctl set-version <version> which can be used in any part and can be picked up by àdopt-info`.

Lin-Buo-Ren · 2018-11-11 16:26:36 UTC

I would like to use the version: git like version string, but instead of describing the recipe containing repo, make it describe the part’s source repo according to the adopt-info key’s value. The described version is enough for me and I want to avoid calling snapcraftctl set-version <version> when possible.

Lin-Buo-Ren · 2018-11-11 16:29:39 UTC

I’ve encountered a problem that in snapcraft 2.43.1(snap) the snapcraftctl command can’t be called in a script called in a override-pull scriptlet:

Pulling hello 
Cloning into '/root/build_hello-snapcrafters/parts/hello/src'...

...

/root/build_hello-snapcrafters/parts/hello/src/../../../snap/local/utilities/set-snap-version.bash: line 35: snapcraftctl: command not found
Failed to run 'override-pull': Exit code was 127.

This issue isn’t reproduced in build.snapcraft.io, nor it is reproduced in snapcraft 3.0

Lin-Buo-Ren · 2019-01-25 21:20:53 UTC

According to the setuptools documentation the description key is equivalent to Snapcraft’s summary key and the long_description key is equivalent to Snapcraft’s description key, however currently Snapcraft uses the description key’s value as the snap description and ignores the long_description key in setup.py, I would like to request changing the behavior to match with setuptools.

UPDATE: Bug #1813364 “The setup.py parse-info method treats description/long_description keys wrongly” : Bugs : Snapcraft

sergiusens · 2019-01-28 12:48:34 UTC

Hi, good catch, if you would want to work on this it would be welcomed!

Lin-Buo-Ren · 2019-01-30 19:14:39 UTC

I’d like to, but unfortunately I’m currently not technically capable of doing so as I don’t speak either Python or Go…

sergiusens · 2019-01-31 20:57:21 UTC

@degville hey, I asked this by other means, it may have been missed due to that, the ask is, can we get this on the main docs somehow? When I asked, it wasn’t even searchable (using the same query which yields results here on the forum: “adopt-info”)

degville · 2019-01-31 21:19:30 UTC

…sorry, this must have fallen off my radar. But yes, definitely. I’ll get it added and make sure it’s returned in the search results. Thanks for the prod!

alan_g · 2019-02-01 10:45:12 UTC

It looks to me as though there’s a missing metadata source: deb package information.

It isn’t hard to parse the output from, say, apt-cache policy and use snapcraftctl but I can’t be the only one doing that?

ijohnson · 2019-03-27 19:44:30 UTC

I took a stab at a PR for this here: https://github.com/snapcore/snapcraft/pull/2518

dhoomakethu · 2019-06-14 05:12:37 UTC

I am trying to use scriptlets to set the version for my snap. The build fails every once in a while with Failed to generate snap metadata: 'adopt-info' refers to part 'my-part', but that part is lacking the 'parse-info' property. . I am running snapcraft v3.5 and building on Ubuntu 16.04 with my base set to core. If I clean only the part (python) that is causing this failure and re run snapcraft, the build goes fine and a snap is generated.

Here is my override-pull

override-pull: |
      snapcraftctl pull
      snapcraftctl set-version "$(python3 -c 'from mypackage import version; print(version.__version__)')"

I am referring https://docs.snapcraft.io/using-external-metadata#meta-scriptlet .

Any Idea why is this happening ?

sergiusens · 2019-06-15 19:55:31 UTC

Does this only happen when “re-running” or on first run?

dhoomakethu · 2019-06-15 23:20:34 UTC

Happens on the first run or any clean run, If I clean the part and build again the build goes through.

jamesh · 2019-07-09 08:39:37 UTC

One thing missing from the description of the parse-info key is a description of how the metadata file is located.

Is it expected to be in the part’s src directory? The build directory? Or the install directory? If it searches multiple locations, which takes precedence?
At which step of the build process is the metadata extracted?