Errors will happen when upgrading the libraries

snapd
1

An error will happen when upgrading libraries godbus/dbus and coreos/go-systemd:

For example----

github.com/godbus/dbus

-Latest Version: v5.0.3 (Latest commit 37bf87e on 30 Sep 2019)

-Where did you use it:

https://github.com/snapcore/snapd/search?p=1&q=godbus%2Fdbus&unscoped_q=godbus%2Fdbus

-Detail:

github.com/godbus/dbus/go.mod


module github.com/godbus/dbus/v5

go 1.12

github.com/godbus/dbus/introspect/introspectable.go


package introspect

import (

"github.com/godbus/dbus/v5"

…

)

This problem was introduced since godbus/dbus commit dda416a on 4 Sep 2019 . Now you used the version commit 4481cbc on 26 Jul 2019 . If you try to upgrade coreos/go-systemd to version commit dda416a on 4 Sep 2019 and above, you will get an error— no package exists at "github.com/godbus/dbus/v5"

Similar issues can also happen when upgrading libraries github.com/coreos/go-systemd.

I investigated the libraries’ (godbus/dbus and coreos/go-systemd) release information and found the root cause of this issue is that----

  1. These dependencies all added Go modules in the recent versions.

  2. They all comply with the specification of “Releasing Modules for v2 or higher” available in the Modules documentation. Quoting the specification:

A package that has migrated to Go Modules must include the major version in the import path to reference any v2+ modules. For example, Repo github.com/my/module migrated to Modules on version v3.x.y. Then this repo should declare its module path with MAJOR version suffix “/v3” (e.g., module github.com/my/module/v3), and its downstream project should use "github.com/my/module/v3/mypkg" to import this repo’s package.

  1. This “github.com/my/module/v3/mypkg” is not the physical path. So earlier versions of Go (including those that don’t have minimal module awareness) plus all tooling (like dep, glide, govendor, etc) don’t have minimal module awareness as of now and therefore don’t handle import paths correctly See golang/dep#1962, golang/dep#2139.

Note: creating a new branch is not required. If instead you have been previously releasing on master and would prefer to tag v3.0.0 on master, that is a viable option. (However, be aware that introducing an incompatible API change in master can cause issues for non-modules users who issue a go get -u given the go tool is not aware of semver prior to Go 1.11 or when module mode is not enabled in Go 1.11+).

Pre-existing dependency management solutions such as dep currently can have problems consuming a v2+ module created in this way. See for example dep#1962.

https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Suggested solutions

1. Migrate to Go Modules.

Go Modules is the general trend of ecosystem, If you want a better upgrade package experience, migrating to Go Modules is a good choice.

2. Keep using the dependency manage tool (like dep, glide, govendor, etc).

Because the import paths have different meanings between the projects adopting module mechanism and the non-module mechanism.

If you prefer to use the dependencies manage tool (like dep, glide, govendor, etc), a textbook example provided by repo github.com/moby/moby can prevent this type of issues.

https://github.com/moby/moby/blob/master/VENDORING.md

https://github.com/moby/moby/blob/master/vendor.conf

In the vendor directory, github.com/moby/moby adds the /vN subdirectory in the corresponding dependencies.

This will help more downstream modules to work well with your package, otherwise they will be stuck in github.com/coreos/go-systemd v21 and github.com/godbus/dbus v4.1.0.

Do you plan to solve this issue? Which solution do you prefer, 1 or 2?

I can submit a PR to fix it.

Hope this issue report can help you :slight_smile:

Thank you very much for your attention.

Best regards,

Kate

2

Hi. Can you describe the problem you are facing? Are you building snapd youself or as part of a distribution and the dependencies cause issues? Is there a known bug in go-dbus and go-systemd?

Modules support is on our TODO list. However, we need to support some older distributions that are stuck with Go 1.9. Once we have a solution to use a newer Go everywhere, modules support will be proposed.

For now we use govendor to manage dependencies. Downstream distributions are free to make their own choices whether to use the vendored packages or try something else.

1 Like
3

Hi，Thanks for your reply. I used snapd as lib and I used Go Modules. Then I found the indirect dependency “godbus/dbus” was stuck in version v4.1.0.

module gomodproject7
go 1.13
require (
	github.com/godbus/dbus v4.1.0+incompatible // indirect
	...
)

But snapd used the version above v5.0.1 (github.com/godbus/dbus).
snapcore/snapd/vendor/vendor.json

"path": "github.com/godbus/dbus",
"revision": "4481cbc300e2df0c0b3cecc18b6c16c6c0bb885d",
"revisionTime": "2019-07-26T02:52:47Z"

This is because godbus/dbus have modules in v5+. And the import paths have different meanings between the projects adopting module mechanism and the non-module mechanism.
When I checked the dependencies version, I found this issue. So I send the report. This report is a warm prompt for you to prevent or combat this issue. Just let you know the problem if you upgrade these dependencies in the future. Hope this can help you.
Thanks again.
Kate

4

Is there a particular package within snapd you are trying to import into your Go program? I’m not sure anything outside of github.com/snapcore/snapd/client could be considered stable.

5

snapcore/snapd/asserts and snapcore/snapd/osutil
I see others used too:
CanonicalLtd/serial-vault, stdevHsequeda/FindProductBot, CanonicalLtd/iot-identity, stdevHsequeda/OfertasBot

6

The two stdevHsequeda projects appear to be importing github.com/snapcore/snapd/osutil for a single 4 line function. They would probably be better off copying the function, since there is no guarantee of stability. If you are after a utility function from that package, I’d recommend doing the same.

The serial-vault and iot-identity packages are much more closely linked with the snapd ecosystem (they are there to support Ubuntu Core devices), so they would need to deal with changes to snapd anyway.

7

OK，thank you for your reply.