Is there a way to utilise a bz2 file in the `source` directive?

snapcraft
1

Is there a way to utilize a bz2 file in the source directive?

I’m working on a snap for Restic in which the author prefers that the snap use the “official” binary. But his releases come out as a bz2 file. When targeting the file I get an error stating:

raise ValueError(‘no handler to manage source ({})’.format(source))
ValueError: no handler to manage source (https://github.com/restic/restic/releases/download/v0.8.1/restic_0.8.1_linux_amd64.bz2)

So if I am reading that correctly - my assumption is that bz2 is either unsupported or requires some type of additional config in the YAML file.

EDIT - I do realise that I could likely pull the file down and use various directives to prep the source locally. Just hoping to keep the yaml as dirt simple as possible.

EDIT2 - Ok, found many examples of people using bz2 files. So I must be doing something else wrong. And I am guessing that it might be because there is a binary in the bz2 file and not the source.

name: restic # you probably want to 'snapcraft register <name>'
version: '0.8.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Fast, secure, efficient backup program. # 79 char long summary
description: |
  "Restic is a program that does backups right. The design goals are :
  - Easy: Doing backups should be a frictionless process.
  - Fast: Backing up your data with restic should only be limited by your
  network or hard disk bandwidth.
  - Verifiable: Much more important than backup is restore, so restic
  enables you to easily verify that all data can be restored.
  - Secure: Restic uses cryptography to guarantee confidentiality and
  integrity of your data. The location the backup data is stored is
  assumed not to be a trusted environment.
  - Efficient: With the growth of data, additional snapshots should
  only take the storage of the actual increment.
  - Free: restic is free software and licensed under the
  BSD 2-Clause License and actively developed on GitHub."

confinement: devmode # use 'strict' once you have the right plugs and slots
grade: devel

parts:
  restic:
    # See 'snapcraft plugins'
    plugin: dump
    source: https://github.com/restic/restic/releases/download/v$SNAPCRAFT_PROJECT_VERSION/restic_$SNAPCRAFT_PROJECT_VERSION_linux_amd64.bz2
    source-type: tar

apps:
  restic:
    command: restic

Just for reference. This is not yet complete as I need this snap to be “classic” I believe as it will need to access the entire file system (backup tool).

EDIT3 - Further in. Current error (updated my YAML file above):

tarfile.ReadError: file could not be opened successfully

EDIT4

So I tested that on the download…

➤ tar xvjf restic_0.8.1_linux_amd64.bz2                                                           19:53:28
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

But then realised that tar can only handle .tar files with or without compression, but not non tar’ed archives.

So back to my original question - can we use bz2 files (that are not tar’s)?

Thanks, all.

2

Alright - here is what I ended up with that got me a working devmode version… Any red flags in the way I did it? Maybe a more optimised way?

SNAPCRAFT.YAML

name: restic # you probably want to 'snapcraft register <name>'
version: '0.8.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Fast, secure, efficient backup program. # 79 char long summary
description: |
  "Restic is a program that does backups right. The design goals are :
  - Easy: Doing backups should be a frictionless process.
  - Fast: Backing up your data with restic should only be limited by your
  network or hard disk bandwidth.
  - Verifiable: Much more important than backup is restore, so restic
  enables you to easily verify that all data can be restored.
  - Secure: Restic uses cryptography to guarantee confidentiality and
  integrity of your data. The location the backup data is stored is
  assumed not to be a trusted environment.
  - Efficient: With the growth of data, additional snapshots should
  only take the storage of the actual increment.
  - Free: restic is free software and licensed under the
  BSD 2-Clause License and actively developed on GitHub."

confinement: devmode # use 'strict' once you have the right plugs and slots
grade: devel

parts:
  restic:
    # See 'snapcraft plugins'
    plugin: dump
    source: .
    prepare: |
      chmod +x prep/extractor.sh
      prep/extractor.sh
    organize:
      restic: bin/restic

apps:
  restic:
    command: bin/restic

EXTRACTOR.SH

#!/usr/bin/env bash

WRK=$(pwd)
cd $WRK

# Had some oddities concatting the strings.
VER="v"
VER+="$SNAPCRAFT_PROJECT_VERSION"
FILENAME="restic_"
FILENAME+="$SNAPCRAFT_PROJECT_VERSION"
FILENAME+="_linux_amd64"

wget https://github.com/restic/restic/releases/download/$VER/$FILENAME.bz2
bunzip2 $WRK/$FILENAME.bz2
mv $WRK/$FILENAME $WRK/restic
chmod +x $WRK/restic

exit 0
3

Hit here,

Here’s a quick rudimentary solution (optimizations could be done on file handling) using local plugins:

$ cat snap/snapcraft.yaml 

name: restic # you probably want to 'snapcraft register <name>'
version: '0.8.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Fast, secure, efficient backup program. # 79 char long summary
description: |
  "Restic is a program that does backups right. The design goals are :
  - Easy: Doing backups should be a frictionless process.
  - Fast: Backing up your data with restic should only be limited by your
  network or hard disk bandwidth.
  - Verifiable: Much more important than backup is restore, so restic
  enables you to easily verify that all data can be restored.
  - Secure: Restic uses cryptography to guarantee confidentiality and
  integrity of your data. The location the backup data is stored is
  assumed not to be a trusted environment.
  - Efficient: With the growth of data, additional snapshots should
  only take the storage of the actual increment.
  - Free: restic is free software and licensed under the
  BSD 2-Clause License and actively developed on GitHub."

confinement: devmode # use 'strict' once you have the right plugs and slots
grade: devel

parts:
  restic:
    # See 'snapcraft plugins'
    plugin: x-restic
    source: .

apps:
  restic:
    command: bin/restic

And the local plugin

$ cat snap/plugins/x_restic.py

# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
import bz2
import os

import requests

import snapcraft


class ResticPlugin(snapcraft.BasePlugin):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.source = ('https://github.com/restic/restic/releases/download/v0.8.1/'
                       'restic_0.8.1_linux_amd64.bz2')
        self.download_path = os.path.join(self.sourcedir, 'restic.bz2')
        self.extract_path = os.path.join(self.builddir, 'restic')
        self.install_path = os.path.join(self.installdir, 'bin', 'restic')

    def enable_cross_compilation(self):
        pass

    def pull(self):
        super().pull()
        r = requests.get(self.source)
        with open(self.download_path, 'wb') as f:
            f.write(r.content)
        
    def build(self):
        super().build()
        with bz2.open(self.download_path, 'rb') as fr:
            with open(self.extract_path, 'wb') as fw:
                fw.write(fr.read())
        os.chmod(self.extract_path, 0o755)
        os.mkdir(os.path.dirname(self.install_path))
        os.link(self.extract_path, self.install_path)
4

@sergiusens - That’s pretty interesting. Is there a benefit to doing it this way over the shell script I had used? I mean it keeps your versions YAML much cleaner.

5

One other quick question… with plugins, can you leverage things like $SNAPCRAFT_PROJECT_VERSION?