Classic confinement review: rescript

Thank you for your quick answer, I didn’t knew about that. Sorry to bother but I don’t get what I have to do in my .yaml file for this to work. So far I’ve tried this and it doesn’t work (this is my first time working with a snap, sorry for the noob question):

plugs:
  personal-files: 
    write: [ $HOME/.rescript/*/* ]

It build okay but when installing I’m getting this error:

2019-01-26T17:30:17-05:00 INFO snap "rescript" has bad plugs or slots:
personal-files (unknown interface "personal-files")

Is it the snap version? Because in the link you posted says: “Requires snap version 2.37+”. If it is the snap version, how can I update it? I can use the snap package but it doesn’t work with the $HOME files indicated. Thanks again and sorry if my question is too basic; again, I’m new at this.

~$ snap version
snap    2.36.3
snapd   2.36.3
series  16
ubuntu  18.10
kernel  4.18.0-13-generic

You’re right in that this interface requires snapd version 2.37+. This should be released to stable soon. However, if you want to test it now, you can by refreshing core on your machine to the beta channel.

snap refresh core --beta

name:      core
summary:   snapd runtime environment
publisher: Canonical✓
contact:   snaps@canonical.com
license:   unset
description: |
  The core runtime environment for snapd
type:         core
snap-id:      99T7MUlRhtI3U0QFgl5mXXESAiSwt776
tracking:     stable
refresh-date: 39 days ago, at 04:06 GMT
channels:                                        
  stable:    16-2.36.3               (6130) 93MB -
  candidate: 16-2.37                 (6259) 95MB -
  beta:      16-2.37                 (6259) 95MB -
  edge:      16-2.37+git1112.cf8d340 (6325) 93MB -
installed:   16-2.36.3               (6130) 93MB core
1 Like

I installed the beta version and it is still not working. With the classic confinement works great but as you can see, using personal-files as described in my past comment did not work. I did a test typing bash -x rescript config to see what it does and using it that way works with the files intended for some strange reason… but using it as it is supposed to be used it doesn’t work and apparently it seems to consider $HOME as /home/user/snap/rescript instead of the user’s home directory and I would be okay with that but the thing is that when using config command with my script you follow a menu to create a configuration file; then when you try to open the menu it doesn’t work for some reason. It seems that the script struggle with the snap directory and it’s own directories that the script will always recognize as /home/username/.rescript. I can upload the snap package generated with this changes here if you want.

I tried again setting up the script to work with ~/snap/rescript/current as the $HOME so it will save and look just there and it doesn’t work either when you try to open the configuration file with the selected editor. The code to open the configuration/exclusions files is "$EDITOR" $HOME/.rescript/config/$conf.conf 2> /dev/null; $conf is determined by read. grep and sed are used to display the names of the configuration/exclusions files. Once you have selected the EDITOR to work with rescript, the selected text editor name is saved in a file and the EDITOR retrieve it using cat $HOME/.rescript/config/.editor. Using bash -x rescript config again:

++ cat /home/sulfuror/snap/rescript/current/.rescript/config/.editor
+ EDITOR=mousepad

So the script is setting the editor right but once you try to open the configuration files (without using bash -x) it doesn’t do anything.

I ended up with my snapcraft.yaml like this:

name: rescript
base: core18
version: '3.7'
summary: Rescript bash script for restic
description: |
  Rescript is a bash script created to manage backups made with
  restic. Restic is a backup program which allows saving multiple
  revisions of files and directories in an encrypted repository.
grade: stable
confinement: strict
architectures: [ all ]
apps:
  rescript:
    command: bash $SNAP/rescript
    environment:
      HOME: /home/$USER
plugs:
  personal-files:
    write: [ $HOME/.rescript ]
parts:
  rescript:
    source: https://gitlab.com/sulfuror/rescript.sh.git
    plugin: dump

With these changes when running the script for the first time typing rescript config this is the output:

cat: /home/sulfuror/.rescript/config/.editor: Permission denied

======================
  Select Text Editor  
======================
 [1] Nano             
 [2] Vim              
 [3] Gedit            
 [4] Mousepad         
 [5] Leafpad          
 [6] Pluma            
 [7] Kate             
 [8] Other            
 [9] Exit             
======================
Select the Text Editor you want to use [ 1 - 9 ]: 1
/snap/rescript/x1/rescript: line 186: /home/sulfuror/.rescript/config/.editor: Permission denied
You have selected [Nano] as your default text editor.
Please type [rescript config] again to set/edit
your configuration/exclusion files.

So the script doesn’t have permissions to access to its files. I tried again changing the configuration files to ~/snap/rescript so maybe that way it could access and manage the files but it doesn’t work. The snapcraft.yaml stayed the same this way except that environment and plugs that are not needed that way. This way, when running the script like this, nano, which is my text editor, doesn’t open configuration files located in /home/sulfuror/snap/rescript/current/.rescript/config/ne.conf. When I use ls -la:

total 24
drwxrwxr-x 2 sulfuror sulfuror 4096 Jan 28 21:31 .
drwxrwxr-x 5 sulfuror sulfuror 4096 Jan 28 21:31 ..
-rw-rw-r-- 1 sulfuror sulfuror    9 Jan 28 21:33 .editor
-rwx------ 1 sulfuror sulfuror 2063 Jan 28 21:31 ne.conf
-rw-rw-r-- 1 sulfuror sulfuror   32 Jan 28 21:31 ne-datefile
-rw-rw-r-- 1 sulfuror sulfuror  349 Jan 28 21:31 ne-exclusions

When I open manually the configuration file using nano ~/snap/rescript/current/.rescript/config/ne.conf it open a blank page. The same with all files. However, if go to the location with my file manager and double click the file, it opens and I can do changes and save the changes.

The worst part about this is that this is a restic wrapper and if I modify the files manually, when I try to do anything, for example, listing snapshots in a repository already created, this is the output:

cat: /home/sulfuror/.rescript/config/.editor: Permission denied
/snap/rescript/x1/rescript: line 1089: /home/sulfuror/.rescript/config/rpi.conf: Permission denied
touch: cannot touch '/home/sulfuror/.rescript/lock/rpi.lock': Permission denied
/snap/rescript/x1/rescript: line 1835: restic: command not found

This is why I think the script needs to be in “classic confinement”, because it just doesn’t work with it’s own files but it also deal with other tools like text editors, sed, cat, grep, it creates configuration files and manage them, a lock file to block other processes and logs; and the most important thing, restic which is distributed as a standalone binary. My guess is that using strict will require me to change a lot of things in the script and I will ended up maintaining two sets of the same script, which is not ideal at all. The code is public, BSD license, free and it is basically “confined” to read/write its own directory only, which always will be at the user’s home directory.

Did you connect the interface? See snap interfaces rescript

Slot   Plug
:home  rescript
-      rescript:personal-files

This is what it shows when running snap interfaces rescript. I don’t know if I’m doing something wrong here. I changed this time the snapcraft.yaml file to this:

name: rescript
base: core18
version: '3.8'
summary: Rescript bash script for restic
description: |
  Rescript is a bash script created to manage backups made with
  restic. Restic is a backup program which allows saving multiple
  revisions of files and directories in an encrypted repository.

grade: stable
confinement: strict
architectures: [ all ]
apps:
  rescript:
    command: bash $SNAP/rescript
    plugs:
      - home
      - personal-files
    environment:
      HOME: /home/$USER
plugs:
  personal-files:
    write: [ $HOME/.rescript, $HOME/.rescript/config/.editor ]
parts:
  rescript:
    source: .
    plugin: dump

Any help would be appreciated. Thanks!

Sorry for the delay. The above indicates that at the time of testing you didn’t have a new enough core snap and the interface is not yet connected (it is manually connected). Please look at snap version and make sure you have snapd 2.37. Then perform:

$ sudo snap connect rescript:personal-files

and report back if this works for you.

PS - the the personal-files interface work, you only need to specify $HOME/.rescript and not files in subdirectories.

Thanks to you for your answer! Basically, doing that at least it displays the configuration files located at $HOME/.rescript but it doesn’t do anything else. This is the output of snap interfaces rescript:

Slot             Plug
:home            rescript
:personal-files  rescript

But this is the output when I run, for example, rescript backup:

==========================================================================
| - - - - - - - - > [ S T A R T I N G    B A C K U P ] < - - - - - - - - |
==========================================================================
Date and Time: Sat Mar 02 2019 10:07:22 PM
/snap/rescript/x1/rescript: line 122: lsb_release: command not found
System: 
Hostname: Hostname
Backup Destination: Server
--------------------------------------------------------------------------
/snap/rescript/x1/rescript: line 1360: restic: command not found
--------------------------------------------------------------------------
End: Sat Mar 02 2019 10:07:22 PM
Duration: 0 seconds
==========================================================================
| - - - - - - - - - > [ B A C K U P      E N D E D ] < - - - - - - - - - |
==========================================================================

So there are a couple of tools that rescript as a snap cannot reach. This is basically a wrapper for restic, which is a backup tool and it is already available as snap, so I can make it a dependency for the script, but restic needs to read all files in the system because, well, that’s what backup tools are for. This script only works with these files mentioned, but restic also have a function to mount via fuse the repository. I made this specific function in my script to create a new temp file to mount the repository and it mounts it in $HOME, so I don’t know how to work with this specific temp directory because it doesn’t exists until you execute rescript mounter. Also, some people will have restic pre-installed with the official binary, so I don’t think this script will be able to locate the binary outside /snap/bin. It also uses rsync in a function called archive; this function restores two snapshots and sync deleted files from one directory to another and then takes another snapshot of these changes (this all happens in /tmp so that’s another plug). So rescript plays a lot with a different tools like echo, sed, awk and others.

Last thing I did just now was to use system-files plug giving access to read in /bin, /usr/bin, /usr/sbin, /usr/local/bin, /usr/local/sbin and it does the same and write in /home, /etc/rescript, /tmp and it does the same with the same errors with the interfaces already connected (sorry about that, I really am a newby in this whole thing).

Slot             Plug
:home            rescript
:personal-files  rescript
:system-files    rescript

Thanks again and sorry if I’m missing something obvious here. And don’t worry about the delay, I know you guys must be busy working in more important things. Thanks for the snaps!

IIUC, your snap can be made strict because it only backs up things that are allowed by strict confinement. Correct?

If so, you bundle restic, rsync, etc in your snap as stage-packages (or build from source), but don’t expose them as snap commands that show up in /snap/bin. In this manner, you can ship whatever restic or rsync you want and rescript calls these rather than system binaries and there is no conflict.

As for using system-files with /usr/* that won’t work the way you want because your snap doesn’t use the host’s /usr but instead a snap-runtime. Instead, ship restic and rsync as suggested and call into $SNAP/path/to/restic (or use Snap layouts)

@sulfuror - your response is needed to proceed with this request.

Closing this request for now. If/when the requester responds, we can pick this up.

I’m sorry for the delay.

What do you mean with “only backs up things that are allowed by strict confinement”? This script only wraps restic tool to backup and restic (also available as snap, so I don’t need to build from source) need access to any file the user is backing up. You can backup user’s home or you can backup your system, depending on the user’s need. This script help you build configuration files for one or more repositories using restic, and also contains “exclusion templates” for home and system backup. AFAIK, restic snap needs classic confinement because of its nature. I don’t know if this script could work without classic confinement and that’s what I’m trying to figure out but no matter what I do, it just wont work. I apologize if I’m missing something obvious here, I don’t really have a lot of knowledge about this.

Ok, this makes the request much more clear. Does the snap only provide backup functionality or does it also provide restore functionality? I ask because a backup interface is in the works: https://github.com/snapcore/snapd/pull/6436 and may suit your snap’s needs with strict confinement if it only backs up.

It can be used to backup and restore. Basically, all restic functions/commands can be used with this script but I have created a couple of functions to make it a little easier to manage. For example, I have wrapped up the forget command in restic, which is used to delete existing snapshots or apply retention policies, in one “rescript” command called cleanup. This command execute restic forget ... followed by an array of policies already set in the rescript configuration file. For restore, it also have a command called restorer which can be used to automatically restore the latest snapshot, or any snapshot without the hassle of typing a long command.

The requirements are understood. Thanks!

@advocacy - can you perform the vetting?

@advocacy - can you please perform the vetting?

@advocacy - ping again, can someone please perform the vetting?

Vetting done, +1 from me for rescript.

Granted use of classic. This is now live.

Your next upload should pass automated review. You may also want to request a manual review for revision 1 in the store if you prefer.