`bind-ssh` not working anymore on Ubuntu 20.04

I’m using the catkin-tools plugin with a .rosinstall file to clone my dependencies from private Github repos using ssh. As discussed here, --bind-ssh works perfectly fine on my Ubuntu 18.04 machine. However, when I’m trying to build the snap from Ubuntu 20.04, I don’t get access:

Initializing workspace (if necessary)...
Merging /root/parts/workspace/src/local/snap.rosinstall
Updating workspace...
Error updating workspace: Cloning into '/root/parts/workspace/src/src/T_core'...
Cloning into '/root/parts/workspace/src/src/Nav'...
Warning: Permanently added the RSA host key for IP address '142.82.121.4' to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Cloning into '/root/parts/workspace/src/src/ads_catkin'...
Cloning into '/root/parts/workspace/src/src/external/benchmark_catkin'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Cloning into '/root/parts/workspace/src/src/external/catkin_simple'...
Cloning into '/root/parts/workspace/src/src/external/ceres_catkin'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Cloning into '/root/parts/workspace/src/src/external/glog_catkin'...
Cloning into '/root/parts/workspace/src/src/external/gflags_catkin'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Cloning into '/root/parts/workspace/src/src/external/suitesparse'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
ERROR in config: Error processing 'Nav' : [Nav] Checkout of git@github.com:ethz/Nav.git version feature/improved_joystick_mapping into /root/parts/workspace/src/src/Nav failed.
Error processing 'T_core' : [T_core] Checkout of git@github.com:ethz-msrl/T_core.git version master into /root/parts/workspace/src/src/T_core failed.
Error processing 'ads_catkin' : [ads_catkin] Checkout of git@github.com:ethz-msrl/ads_catkin.git version 6cb4104 into /root/parts/workspace/src/src/ads_catkin failed.
Error processing 'external/benchmark_catkin' : [external/benchmark_catkin] Checkout of git@github.com:ethz-asl/benchmark_catkin.git version 5018b4b into /root/parts/workspace/src/src/external/benchmark_catkin failed.
Error processing 'external/catkin_simple' : [external/catkin_simple] Checkout of git@github.com:catkin/catkin_simple.git version 0e62848 into /root/parts/workspace/src/src/external/catkin_simple failed.
Error processing 'external/ceres_catkin' : [external/ceres_catkin] Checkout of git@github.com:ethz-asl/ceres_catkin.git version 3f0643f into /root/parts/workspace/src/src/external/ceres_catkin failed.
Error processing 'external/gflags_catkin' : [external/gflags_catkin] Checkout of git@github.com:ethz-asl/gflags_catkin.git version fc38fc5 into /root/parts/workspace/src/src/external/gflags_catkin failed.
Error processing 'external/glog_catkin' : [external/glog_catkin] Checkout of git@github.com:ethz-asl/glog_catkin.git version 1b8eab2 into /root/parts/workspace/src/src/external/glog_catkin failed.
Error processing 'external/suitesparse' : [external/suitesparse] Checkout of git@github.com:ethz-asl/suitesparse.git version 316eb71 into /root/parts/workspace/src/src/external/suitesparse failed.
sed: can't read /root/parts/workspace/src/src/Nav/nav_launch/launch/nav.launch: No such file or directory

It even fails getting the public repositories over ssh. Of course, I already tried cloning the repositories manually by using ssh, which works perfectly fine. And ssh -T git@github.com Hi username! You've successfully authenticated, but GitHub does not provide shell access., which should proof that "Error: Permission denied (publickey)". is not the case.

Any ideas @sergiusens?

This is unfortunate!

A few questions and things to run:

  • How are you invoking Snapcraft?
  • Output of snapcraft version?
  • Did the base for your project change between your switch?
  • Is ~/.ssh available in the build environment? (snapcraft pull --shell). Does it have the right permissions and correct keys?

SNAPCRAFT_BUILD_ENVIRONMENT_MEMORY=8G snapcraft --bind-ssh --debug

4.2

I’m still trying to build for core18, so: base: core18

There is a .ssh directory with the exactly same keys as in the “normal” environment. But shouldn’t it be snapcraft pull --shell --bind-ssh?

snapcraft-nav ../.ssh# ls -l github_key
-rw------- 1 root root 3389 May 25 09:11 github_key

Do you have an ssh config (~/.ssh/config) with something along the lines of:

Host github.com
    User git
    IdentityFile ~/.ssh/github_key

I do have a config file, but just with some lines I added manually for my jump host and not for github.

1 Like

For the config that is passed through, try setting the config for github and see if that works out for you… though I’m not sure why the behavior would be different between 18.04 and 20.04 unless your configs are different?

What is passed through? The whole .ssh folder? Just to understand, is this folder copied to the build space or somehow linked if I use --bind-ssh?

My configs are identical on both systems. But, indeed, adding the github host to the file works. But why is this necessary?

The .ssh folder is mounted inside the build environment, so I supposed shared would be a better description.

It tells ssh to use your github key when connecting to github. Otherwise it may try guessing based on keys in your folder (keychain?), but that’s not reliable for various reasons. I generally have a config section for each ssh host I connect to to prevent issues.

1 Like

Ok, makes sense. Seems like I was just lucky that it always worked so far. I’m wondering why none of the basic tutorials (like the one on Github to generate the SSH keys) are mentioning this.

I assume this is not a snapcraft bug and I can close this now?

1 Like

Correct, unless you find some other discrepancy. Have a good day!