Is there any plan to convert snapd to use Go Modules instead of govendor?
The main reason we haven’t switched yet was that we wanted snapd to remain buildable on Ubuntu 14.04 and 16.04, so were using Go 1.9 as a baseline. That does not support modules, so we continued to use govendor.
As of January 15th, we have a Go 1.13 build for 16.04. So a good first step would be to start using it for 16.04+ builds.
I’m not sure what we’d want to do about 14.04, but the promises under Extended Security Maintenance are fairly limited already, and I’m not sure how updates are currently being handled there.
Looking at the other distros we test on:
- CentOS 7/Amazon Linux 2 look like they have Go 1.15 available via EPEL.
- Debian 9 has Go 1.11 available via backports. If we switched to testing Debian 10 (which will likely become the old stable release later this year), we’d have Go 1.14. Either way, we’d have module support.
- openSUSE 15.1 has Go 1.14 available via updates.
So those aren’t necessarily holding us back either.
Once we’re building on new enough Go versions, it would be possible to try switching to the module based build process. That’s likely to be a bigger job though, so I’m not sure when it is likely to be attempted. If there was some way to build a govendor vendor.json from the go.mod/go.sum files, then that might help with the transition.
Jamesh-
Thanks for the response. Of course I had not considered the legacy support required. That makes a lot of sense.
Would it be crazy to think of using snaps to “self-host” building a new snapd based on modules?
I mean if you have some version of snapd available you can install the go snap which is current and then build the next version that support modules.
It’s not really an option when it comes to getting snapd into other distributions. While we might be able to add snaps to the build process for Ubuntu packages, it’s unlikely that we’d be able to do the same for Fedora’s, Debian’s, openSUSE’s, etc build services.
If the only way they can build new versions of snapd is by rearchitecting their build system, then they’ll probably opt not to package new versions of snapd. But as I said above, we’re probably at a point where we can move to a newer baseline that supports modules.
With that said, I don’t think there is any roadmap for when the change will be made. The work would have no user visible change, so we’d need to weigh up spending time on this task over work that directly benefits users.
It is something that we will need to address though, as it is clearly technical debt. We can’t expect the rest of the Go community to remain compatible with the old versions, so we will eventually need a newer Go in order to upgrade some dependency.
We spoke about this in our daily standup, and I have a branch of snapd which mostly builds using a go.mod that was somewhat imported from the vendor.json, however the larger issue I think is the tooling around building debian/traditional linux packages using go.mod, the debian helpers at least seem to be in different stages of supporting go modules on different releases, so that would take some wrangling to get everything working there at the packaging level, from a code perspective we could probably pull off a transition to using only Go modules in a fairly small PR, but the packaging changes are what will bog us down.
But note that we have a deadline before we need to do this, since we want to be able to build on Arch and Fedora, so when those distros switch to Go 1.17 which drops GOPATH support entirely we need to have switched to using Go modules anyways (at least without any crazy hacks like dynamically generating a go.mod for certain distros when building on them, etc.)
1 Like
Oh, that’s exciting. I was not tracking that 1.17 was dropping GOPATH altogether. That may not be optimal for everyone but is certainly encouraging to me. 
I’ve built snapd (2.48) with Go modules but have not tested anything other than a trivial build of that. It did have some weird issues around tests with empty go.mod files and a udev package that was renamed.
Aside from the govendor “issue” the code base is quite nice to work in and is, in my opinion, high quality and impressively tested.
I was thinking more of building the vendor.json from go.mod/go.sum rather than the reverse: the idea being to treat the new system as the source of truth and derive the govendor support from that. If that’s not possible, having some script to verify that the two are in sync would probably do the trick.
If we get to a point where snapd can be built correctly in both the new and old worlds, then we don’t need to switch all the packaging over in one go. At least until 1.17, we can use GO111MODULE=off to have the Go toolchain ignore the go.mod file for the cases where packaging hasn’t been ported forward. That would mean we don’t need to port everything in one go.
ah that’s an interesting proposal, I don’t think there exists such a tool, but probably we could hand-write one in the snapd tree for this, it seems a fairly simple protocol to just write the SHA’s that go.mod has for the versions it calculated at a given point in time.
Why is osutil/udev actually github.com/pilebones/go-udev?
That was one of the hiccups I ran into when trying to use Go modules.
Well we needed to fork that library to fix somethings, but we didn’t want to upstream it in all the different distros, so vendoring it into snapd proper with a different go import name was the most expedient solution at the time. I think we can clean that up a bit when we use go modules since there is the replace directive which would enable us to rename it at the top level.
1 Like
Depending on how long we expect the transition to take, it might not even be necessary to try and build one dependency file from the other. We don’t change the dependencies particularly often, so simply checking that they’re consistent would probably be sufficient.
1 Like
What’s the status on this? Has it been planned yet?