Snapcraft clean fails with go modules


#1

Hi,

I’m building a snap using snapcraft stable channel, and I’m unable to run snapcraft multiple times without manually cleaning the parts directory. This is because the snap builds multiple parts using go modules. The issue I run into is that after initially building the parts and the snap successfully, if I then go to clean with snapcraft clean, it immediately fails because the files that are staged into the parts dir that are built using go modules have their $GOPATH/pkg/mod (which is inside the parts dir for each go module part) as read-only. This causes snapcraft to fail with a traceback like this:

Traceback (most recent call last):
  File "/snap/snapcraft/2650/legacy_snapcraft/bin/snapcraft", line 11, in <module>
    load_entry_point('snapcraft==2.43.1', 'console_scripts', 'snapcraft')()
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/click/core.py", line 1043, in invoke
    return Command.invoke(self, ctx)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/cli/_runner.py", line 114, in run
    ctx.forward(lifecyclecli.commands["snap"])
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/click/core.py", line 553, in forward
    return self.invoke(cmd, **kwargs)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/cli/lifecycle.py", line 209, in snap
    _execute(steps.PRIME, parts=[], pack_project=True, output=output, **kwargs)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/cli/lifecycle.py", line 98, in _execute
    lifecycle.execute(step, project_config, parts)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 90, in execute
    executor.run(step, part_names)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 194, in run
    self._handle_step(part_names, part, step, current_step, cli_config)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 233, in _handle_step
    self._handle_dirty(part, current_step, dirty_report, cli_config)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 365, in _handle_dirty
    part, hint="({})".format(dirty_report.get_summary())
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 268, in _rebuild
    hint=hint,
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/lifecycle/_runner.py", line 343, in _rerun_step
    part.clean(staged_state, primed_state, step)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 909, in clean
    self._clean_steps(project_staged_state, project_primed_state, step)
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 942, in _clean_steps
    self.clean_build()
  File "/snap/snapcraft/2650/legacy_snapcraft/lib/python3.5/site-packages/snapcraft/internal/pluginhandler/__init__.py", line 651, in clean_build
    shutil.rmtree(self.plugin.build_basedir)
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 474, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 412, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 412, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 412, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 412, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 412, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 412, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 432, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/snap/snapcraft/2650/legacy_snapcraft/usr/lib/python3.5/shutil.py", line 430, in _rmtree_safe_fd
    os.unlink(name, dir_fd=topfd)
PermissionError: [Errno 13] Permission denied: 'systemstat_ex.go'

where systemstat_ex.go is here:

$ ls -lah $(find . -name "systemstat_ex.go")
-r--r--r-- 1 me me 756 Mar 12 08:05 ./parts/edgex-go/build/.gopath/pkg/mod/bitbucket.org/bertimus9/systemstat@v0.0.0-20180207000608-0eeff89b0690/systemstat_ex.go

After reading up on this, go modules pkg explicitly are meant to always only be read-only, see https://github.com/golang/go/issues/27161 for more details on why. I am not sure what the appropriate thing for snapcraft to do is when cleaning like this, but seeing as how the upstream go behavior seems unlikely to change and modules will be default/required for go 1.13+ we should figure out snapcraft ought to handle this situation.

As a workaround, I can usually just run sudo snapcraft clean or sudo rm -rf parts and then it works.

Lastly, if it’s important this snap does not have a base because we need to be able to build this snap inside a docker container and I haven’t had time to investigate how to make our docker setup work with snapcraft bases.

Thoughts? Ping @sergiusens


#2

Are you using bases?


#3

So no I’m not using bases :slight_smile: