Permission denied when building with snapcore/snapcraft

kjackal · September 4, 2018, 5:59am

Hi,

We are using the following to build microk8s on travis

docker run -v `pwd`:`pwd` -v /tmp:/tmp -w `pwd` snapcore/snapcraft sh -c "apt update && snapcraft"

Since yesterday the above is failing with:

Successfully downloaded core at /tmp/tmpt62nav6j/core.snap
Downloading 'core.snap' 100%                                                   
Sorry, Snapcraft ran into an error when trying to running through its
lifecycle that generated a trace that has been put in '/tmp/tmpq4l6lsr0/trace.txt'.
"Submitting this error to the Snapcraft developers is not possible through the CLI
without Raven installed.
If you wish to report this issue, please copy the contents of the previous traceback
and submit manually at https://launchpad.net/snapcraft/+filebug.

And the referenced log is:

Traceback (most recent call last):
  File "/usr/bin/snapcraft", line 9, in <module>
    load_entry_point('snapcraft==2.43', 'console_scripts', 'snapcraft')()
  File "/usr/lib/python3/dist-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1037, in invoke
    return Command.invoke(self, ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/lib/python3/dist-packages/snapcraft/cli/_runner.py", line 93, in run
    ctx.forward(lifecyclecli.commands["snap"])
  File "/usr/lib/python3/dist-packages/click/core.py", line 552, in forward
    return self.invoke(cmd, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/snapcraft/cli/lifecycle.py", line 136, in snap
    project = _execute(steps.PRIME, parts=[], **kwargs)
  File "/usr/lib/python3/dist-packages/snapcraft/cli/lifecycle.py", line 35, in _execute
    lifecycle.execute(step, project_config, parts)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle/_runner.py", line 86, in execute
    project_config.project.deb_arch, project_config.data.get("base", "core")
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle/_runner.py", line 131, in _setup_core
    snap_cache.cache(snap_filename=download_path)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/cache/_snap.py", line 76, in cache
    cached_snap_path = self._get_snap_cache_path(snap_filename)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/cache/_snap.py", line 68, in _get_snap_cache_path
    arch = self._get_snap_deb_arch(snap_filename)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/cache/_snap.py", line 52, in _get_snap_deb_arch
    os.path.join("meta", "snap.yaml"),
  File "/usr/lib/python3.5/subprocess.py", line 626, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.5/subprocess.py", line 693, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied

Is this method of building snaps (ie with docker) supported or should we move to snapcraft cleanbulild?

Thanks

psiinon · September 4, 2018, 7:36am

Getting the same problem here, since yesterday as well.

psiinon · September 4, 2018, 8:35am

For info this is preventing us from releasing zaproxy as stable. We need to use classic confinement but thats failing on the stable and beta docker images with ‘desktop-launch: not found’. Using the latest docker image fixed that, but with this bug we now cant build our snap

Wimpress · September 4, 2018, 10:24am

@sergiusens @kyrofa Please can you take a look at this?

sergiusens · September 4, 2018, 2:40pm

Can you please switch to using the stable tag while we get this sorted?

docker run -v `pwd`:`pwd` -v /tmp:/tmp -w `pwd` snapcore/snapcraft:stable sh -c "apt update && snapcraft"

psiinon · September 4, 2018, 2:48pm

We cant as stable and beta docker images fail with ‘desktop-launch: not found’.
But we havnt released our snap as stable yet so its not breaking something that was working before.

sergiusens · September 4, 2018, 3:23pm

I have created https://bugs.launchpad.net/snapcraft/+bug/1790665 to track this

sergiusens · September 4, 2018, 3:24pm

What version of snapcraft is in your stable docker image where this has failed?

xet7 · September 4, 2018, 9:23pm

How can I revert to previous version of snapcraft?
It seems to not be allowed, and snapcraft snap package is broken, there is permission denied error.

$ sudo snap list
Name                Version  Rev   Tracking  Publisher    Notes
core                16-2.35  5328  stable    canonical✓   core
snapcraft           2.43     1803  stable    canonical✓   classic

$ sudo snap refresh --revision=1802 --classic snapcraft
error: cannot refresh "snapcraft": Access by specifying a revision is not allowed for this Snap.

chipaca · September 5, 2018, 12:29am

you can use revert to roll back to a snap revision you already have. For example:

$ snap list --all snapcraft
Name       Version  Rev   Tracking  Publisher   Notes
snapcraft  2.42.1   1594  stable    canonical✓  disabled,classic
snapcraft  2.43     1803  stable    canonical✓  classic

this means that if I do snap revert snapcraft I’ll get revision 1594. If I had more than one old revision I could use --revision to choose.

See snap help revert for more information; you might lose data when you revert.

xet7 · September 5, 2018, 7:43am

@chipaca

Does not work on Ubuntu 16.04. I am logged into snapcraft.

$ sudo snap list --all snapcraft
Name       Version  Rev   Tracking  Publisher   Notes
snapcraft  2.43     1803  stable    canonical✓  classic

$ sudo snap revert --revision=1594 snapcraft
error: cannot revert "snapcraft": cannot find revision 1594 for snap "snapcraft"

psiinon · September 5, 2018, 8:01am

I’m still getting the same failure when using both of the versions I have installed:

snap list --all snapcraft
Name       Version  Rev   Tracking  Publisher   Notes
snapcraft  2.42.1   1594  stable    canonical✓  classic
snapcraft  2.43     1803  stable    canonical✓  disabled,classic

psiinon · September 5, 2018, 8:03am

FYI our snap is defined here: https://github.com/zaproxy/zaproxy/tree/develop/snap

xet7 · September 5, 2018, 8:26am

Bugfix is at https://github.com/snapcore/snapcraft/pull/2240 , but it’s not at newest snap package.

So I removed snap version, and installed snapcraft from source, and it seems to work for me.

Snapcraft source install on Ubuntu 16.04 64bit

# Add to /root/.bashrc:
export PATH="$PATH:/home/user/repos/snapcraft/bin"

# Install dependencies
sudo apt install python3-yaml python3-tabulate python3-pymacaroons python3-progressbar \
python3-requests-unixsocket python3-petname python3-pyelftools xdelta3

# Install snapcraft
cd ~/repos
git clone https://github.com/snapcore/snapcraft.git
cd snapcraft
sudo python3 setup.py install

# Workaround bug https://bugs.launchpad.net/snapcraft/+bug/1656884/comments/1
sudo ln -s /usr/local/lib/python3.5/dist-packages/snapcraft-2.43-py3.5.egg/share/snapcraft/ /usr/share/snapcraft

# Build package
cd ~/repos/myproject
sudo snapcraft

chipaca · September 5, 2018, 9:05am

that shows you don’t have any other revision, and as I said

(if you don’t have it, you can’t revert to it)

sergiusens · September 5, 2018, 9:59am

Can you detail how you are building? Having the snap and having docker are orthogonal situations for me.

But considering docker, by doing,

diff --git a/snap/build-snap.sh b/snap/build-snap.sh
index 8a73ab6..9b09623 100755
--- a/snap/build-snap.sh
+++ b/snap/build-snap.sh
@@ -1,2 +1,2 @@
-docker pull snapcore/snapcraft
-docker run -it -v "$PWD:$PWD" -w "$PWD" snapcore/snapcraft snapcraft
\ No newline at end of file
+docker pull snapcore/snapcraft:stable
+docker run -it -v "$PWD:$PWD" -w "$PWD" snapcore/snapcraft:stable snapcraft

I successfully built zaproxy, here’s a full video doing that from a fresh Digital Ocean instance using its pre-baked docker setup

If however you are using snapcraft, I do not suspect the title of this topic is what is affecting you and I would appreciate a new thread to not mix up the topics for other casual readers passing through

psiinon · September 5, 2018, 10:19am

I’m building a snap because we want to release ZAP as a snap. I’m using docker to build it because I use Fedora which is not supported for building snaps

Yes there are 2 different problems here. Although you can build zaproxy with snapcraft:stable you cannot run it - it fails with ‘desktop-launch: not found’. Thats a known problem which appears to happen for both stable and beta docker containers. Thats why we need to use latest. If you use latest then the build fails as per this issue. Any idea when the latest image will be rebuilt?

xet7 · September 5, 2018, 11:30am

This affects also me, because I get permission denied error.

Docker Hub image snapcore/snapcraft installs old version of snapcraft from Ubuntu xenial repos:

To make new Docker version, you would need Dockerfile similar to this, but completed with all steps (I did not have time to finish it yet)

FROM ubuntu:xenial

# Enable multiverse as snapcraft cleanbuild does.
RUN sed -i 's/ universe/ universe multiverse/' /etc/apt/sources.list

RUN apt-get update && \
  apt-get dist-upgrade --yes && \
  apt-get install --yes \
  git \
  python3-yaml \
  python3-tabulate \
  python3-pymacaroons \
  python3-progressbar \
  python3-requests-unixsocket \
  python3-petname \
  python3-pyelftools \
  xdelta3 \
  && \
  git clone https://github.com/snapcore/snapcraft.git && \
  cd snapcraft && \
  python3 setup.py install && \
  ln -s /usr/local/lib/python3.5/dist-packages/snapcraft-2.43-py3.5.egg/share/snapcraft/ /usr/share/snapcraft && \
  # ADD MORE STEPS HERE
  && \
  apt-get autoclean --yes && \
  apt-get clean --yes

The snap version of snapcraft, that is installed with sudo snap install snapcraft --classic also does not have newest fix for permission denied yet.

I did not get snapcraft working in Docker, VirtualBox, etc. Only way to get it working for me was to install Xubuntu 16.04 64 desktop, and then install snapcraft from GitHub repo, so I did get Wekan snap package built and uploaded to snap store.

To remind myself later, I added wiki page for myself to:

Today I noticed new activity on this issue, so I hope snap build servers will be fixed:

sergiusens · September 5, 2018, 11:35am

For cleanbuild? That is true, 2.43.1 should have the fix, snapd was released a bit to quickly for us to sync that out.

Why don’t you docker pull snapcore/snapcraft:stable though? That dockerfile, as is, will miss out environment settings when building classic confined snaps.

xet7 · September 5, 2018, 11:44am

I did try docker longer time ago, and it did not work for me. Also using virtualbox did not work for me.

I build wekan for strict confinement. Not classic.

When installing snapcraft, sudo snap install snapcraft --classic installs snapcraft command for building snap packages locally with command sudo snapcraft on my local desktop. It does run on my laptop bare metal, so command line option cleanbuild is not used. But that snap package version of snapcraft command is broken, so I installed snapcraft command from source, without any docker/virtualbox/lxc.