Snapcraft version scripts


#1

:warning: This is depreacated. Refer to Extracting information from sources in snapcraft parts

There has been high desires to be able to tailor the version according to many rules, some of these include:

  • add part of a VCS hash.
  • read the version which is some other file.
  • do some something dynamic like adding the date.

While most of this information can be encoded from the grade and channel installed from, users seem to just want to look at the version. To solve that matter my proposal is to extend the schema to allow

{
    'version-script': {
        'type': 'string'
    }
}

If version-script is selected, then rules for scripts apply and whatever is echoed is what ends up as the version (stripping any trailing newlines). To make things more dynamic, the script will have access to a variable SNAPCRAFT_GRADE with the already existing SNAPCRAFT_PROJECT_VERSION which makes something like this construct possible:

version-script: |
    if [ "$SNAPCRAFT_GRADE" = "stable" ]
    then
        echo $SNAPCRAFT_PROJECT_VERSION
    else
        echo $SNAPCRAFT_PROJECT_VERSION-$(git rev-parse HEAD)

Given that parts can chain together in different ways with the after keyword and we don’t want to complicate the logic for any user, this will run during the creation of meta which happens after all parts have been staged and is only useful for visualization purposes.

Additionally version will allow an entry of git which is a hint for snapcraft to construct a version for you with the following rules for the VCS repo containing the snapcraft project (from running git describe):

  • if HEAD == $latest_tag -> version will be set to $latest_tag.
  • if HEAD has commits ahead of $latest_tag -> version will be set to $latest_tag+gitN.ABCDEF1 where N is the number of commits ahead and ABCDEF1 the commit hash for HEAD.
  • if there is no tag defined -> version will be set to 0+git.ABCDEF1 where ABCDEF1 is the commit hash for HEAD

Welcome snapcraft 2.29
Done: snapcraft 2.29
#2

so how would i dynamically use a specific package version from one of the packages installed inside the snap or read from a file that is created during the build step of my snap ?
i.e. i guess something like the git rev example above is not executed during build but eariler.


#3

That is a good constraint to make it something we do at the end of the build, and then this brings in the constraint that the SNAPCRAFT_PROJECT_VERSION exported anywhere in the environment will always be the one in the version field.

I guess this is ok, but I need to sit on it and probably wait for some other opinions to land.


#4

So, you can have both version and version-scriptlet?


#5

I initially thought, either or. Which makes SNAPCRAFT_PROJECT_VERSION reusable in the version-scriptlet, we can also make this an oneOf but that means we need to compute the version at the beginning of snapcraft.

The other option, would be to do a two pass on the version, but it feels like it would be something rather broken. I am torn here, which is why I have left this task in limbo for so long.


#6

how about introducing a second variable SNAPCRAFT_VERSION_OVERRIDE, that is only used after the build step and overrides in any case. that way you can have both and have a clear separation through the variable naming.


#7

For my use case, it has to run after the pull step at the very earliest. Running after build probably makes most sense though?


#8

How can we use the SNAPCRAFT_PROJECT_VERSION env variable when building on Launchpad?
I guess it’s an env variable that needs to be set before running snapcraft, right?


#9

As I understand it, this is the first thing we’re suffixing “-scriptlet”, which is inconsistent with the others. It’s also confusing; I probably don’t know what a scriptlet is.

May I suggest we just call it version: and interpret it as a script if and only if the value is multiline?


#10

It is possible to script without being multiline, which I think would make it more confusing.


#11

For this case I think we will need to go with @ogra’s suggestion of an override as this would invalidate any use of SNAPCRAFT_PROJECT_VERSION which gets injected into the environment early in the build so it is usable by parts and all the lifecycle steps.


#12

This variable is exported by snapcraft into the environment, so there is nothing to be done by launchpad or any builder.


#13

I’ve wondered the same about the term “scriptlet”.

@sergiusens Do we have other variables today that may be set to a script in snapcraft.yaml, and if so how are they named?


#14

The keywords in the yaml that take a script as value are prepare, build and install.


#15

Yeah, the ones we came up with a while back, prepare, build and install which probably should of been named <step>-prepare, <step>-build and <step>-install where <step> could be pull or build.


#16

Seems easy to establish a better pattern now, while remaining compatible. The -script (no need for let) suffix also sounds like a nice touch to make them alike.

That said, the suggestion above doesn’t seem so interesting… pull-build? pull-install? build-build? Not very intuitive.

How about:

  • version-script
  • pre-pull-script
  • pull-script (runs instead of)
  • post-pull-script
  • pre-build-script
  • build-script (runs instead of)
  • post-build-script
  • pre-install-script
  • install-script (runs instead of)
  • post-install-script

I understand that build+install is done by a single method inside snapcraft’s implementation, but perhaps it should be considered further whether this is a good idea. It’s definitely not something people will expect upfront at least, so perhaps it might be tweaked so the steps above could make sense.


#17

I’ve always wanted to strip out install into its own step. At least it could be a supported mechanism for build systems where they are actually different.

Just want to note that we are dangerously walking into non-declarative land here.

Still, I am on board :wink:


#18

We’ve already crossed that line as you note above, and we did that very consciously.

Constraining the language to being entirely declarative so that it can then call a Makefile that is almost entirely procedural is not a great advantage. We’re making people’s lives more difficult without giving much in return.

That pain hurts more because most build systems don’t have a very strict way of running, which means project documentation ends up with that “how to build” section that needs to be translated into a snapcraft.yaml file. If we are too strict about it being declarative, we’ll then end up with thousands of trivial plugins simply adapting the way the underlying build system is called.

Having tiny scripts inline for that sort of tuning is more comfortable.

That’s the motivation behind allowing small scripts in snapcraft.yaml.


#19

Oh, I completely understand the reasons :slight_smile:


#20

It feels to me that foo-script could easily become overkill, when really what we want in most cases is variable substitution.

I haven’t read the full thread, but have we explored the idea that a scriptlet early in the whole process, which is run every time, can set environment variables (“SNAPCRAFT_FOO”) which are then accessible inside the snapcraft.yaml with $SNAPCRAFT_FOO?

So this means there is ONE scriptlet which is always run if defined, and that sets environment variables, and environment variables can be substituted into snapcraft.yaml. I far prefer this to a xxx-script for anything we might want to substitute.