Permissions problem using snapcraft in Azure Pipelines

Hello folks,

I’ve recently been running into some issues when trying to use snapcraft in an Azure Pipelines CI setup. As far as I can tell it’s related to a permissions problem related to snapd.

Installation works OK (sudo snap install --classic --stable snapcraft) but when the CI job tries to run snapcraft it emits an error:

+ snapcraft --version
/ not root-owned 1001:50
+ snapcraft
/ not root-owned 1001:50

The error is correct: / in the CI job is owned by vsts, not root (which I presume is a special user created by Azure Pipelines).

snapd itself is running as root, as one would expect.

Any advice on how to deal with this? I presume it’s a safety feature to prevent snapd from writing into directories it shouldn’t be able to, but in this case it appears to be coming up against a deliberate decision by Azure Pipelines that the root dir of the CI job should not be owned by the root user. (Most of its contents on the other hand are root-owned: all the usual system dirs.)

Having problems that is similar and likely the same error.

First a step is run with:

sudo apt update
sudo apt install snapd
sudo snap install --classic snapcraft

Then electron-builder is called with the snap flag, and it logs:

snapcraft is not installed, please: sudo snap install snapcraft --classic

Building the .snap itself works fine I think, it is simply the uploading with snapcraft that fails.

likely not … this is just a common mistake and we should somehow make sure some message is displayed from the snapd postinst script or so …

what happens is:

installs snapd and all the tools, creates /snap/bin and adds info about it to the login scripts that create your PATH variable during login …

installs /snap/bin/snapcraft

… does not find /snap/bin/snapcraft because the scripts above that craft your PATH variable only run at login time, so the rule of thumb after you installed the snapd deb is to log out and back in again, which will then make your PATH include /snap/bin and electron-builder will find the snapcraft binary …

this is unrelated to root filesystem permissions …

Thanks for the reply, logout and login is likely not a viable solution for a hosted build on Azure DevOps. Perhaps simply adding the PATH to include /snap/bin as a shell cmd in the process might be enough. Will do some testing and see.

Sorry for belated follow-up here – things got busy with the day job :wink:

My own azure pipelines setup is pretty simple:

- 1.*

- job: ldc2_snap
  timeoutInMinutes: 0
    vmImage: ubuntu-16.04
  - script: |
      set -x
      snap version
      lxd --version
      sudo apt-get update
      sudo snap install --classic --candidate snapcraft
      export PATH="${PATH}:/snap/bin"
      snapcraft --version
    displayName: Build ldc2 snap package

Note that this assumes that snapd is already installed (which seems reasonable and has always worked before), and it exports an updated PATH that includes /snap/bin (so there should be no difficulty finding the installed snapcraft). The problem in this case really does look like it’s down to permissions.

Adding PATH should solve your problem, yes

Your problem is certainly not related to PATH at all, but I will let the snapd team address this issue.

Hi, I have same problem in GitHub Actions:

Run snapcraft login --with ***
/ not root-owned 501:50
##[error]Process completed with exit code 1.

I am seeing the same issue as @guilhem which is blocking my use of snaps in Github Actions.

For some reason it fails in one repository while it seems to work in another repository.

My specific failure is:

Run sudo snap install juju --classic --channel=2.4/stable
2019-11-26T20:55:21Z INFO Waiting for restart...
error: cannot perform the following tasks:
- Run configure hook of "juju" snap if present (run hook "configure": / not root-owned 501:50)

Any workaround would be much appreciated.

The relevant code for this is was merged in, and a basic summary of the reason for this is that snap-confine needs to load a BPF program into the kernel, and to be as safe as possible, we ensure that the directory where these BPF programs are stored all the way back up to “/” is root owned so that only trusted programs (i.e. snapd) can put BPF programs into that directory. We do this because allowing non-root to put arbitrary programs into that directory would allow non-root to effectively load arbitrary code into the kernel for execution. See also the explanation here:

I’m not sure what we should do in snapd to mitigate this… @mvo and @jdstrand any ideas? Perhaps we could have a system flag to selectively disable this on CI environments like the Azure cloud? Maybe it’s “safe enough” to just ensure that all directories below “/” but not including “/” are root-owned?

Is it possible that some kind of Azure Pipelines or GitHub Actions agent that runs when the job starts has some kind of race condition where it will re-own “/” as root? I don’t think snaps would work at all if “/” is not root owned as discussed earlier in this post, so it’s a bit confusing to me why it would sometimes work for you and other times hit this issue.

Could be. I’ll try to drill down what’s going on in a couple of separate test repositories.
Thanks for the relevant code and info. Some kind of safe enough flag would probably be handy here.

This is an important safety measure since otherwise the owning user is able to move aside anything in / and recreate the directories/subdirectories/files to effectively modify anything under /. This sounds like a bug in Azure, cloud-init configuration, something else (eg, chown vsts $TPYO/) and if it is intentional, I would encourage that decision be reconsidered. There are historical CVEs for exactly this sort of thing.


I guess it didn’t occur to me for some reason, but as a work-around until this is fixed upstream with Azure pipelines, could folks try running sudo chown root / since you can use sudo to install a snap ?

Tried this now, doesn’t have any effect (same error). Should I try with recursive flag? mmm

Tried this now, doesn’t have any effect (same error)

Turns out this didn’t work because after the chown command, the group owner was still not root. After trying sudo chown root:root / it seems to work (at least to print the version of snapcraft, will keep testing more stuff now).

Unfortunately that only worked for printing the version of snapcraft (which surprisingly enough is always 3.8 regardless if I pass --stable flag or not). Unfortunately, if I try to build a snap package after that, I get this error:

You need 'multipass' set-up to build snaps:

What does this mean…?

Lo and behold, running snapcraft with --destructive-mode seems to work! I’ll test the snap package that was generated to see if it’s not complete garbage though.

1 Like

By default, snapcraft will try to build with multipass which launches VM’s. As you have found, you can also build “natively” with --destructive-mode, but note that in this case you need to make sure the build-environment that Azure Pipelines sets up for you matches that of the base setting in your snapcraft.yaml, so if your snapcraft.yaml has base: core18, you need the build environment to be Ubuntu 18.04.

Alternatively, you could try having snapcraft launch lxd containers, but this is probably an unnecessary layer of abstraction since Azure Pipelines is already creating a container for your build. To try this, you would just call snapcraft with the --use-lxd option and also install the lxd snap.

Ahoy all – just to say thanks for the chown and --destructive-mode hints: those also work for me :slight_smile:

1 Like

Note that I filed an upstream issue for Azure here: