Handling docs updates on a forum is a bad design choice

snapcraft
1

I recently suggested many documentation updates and the conversation is beginning to look quite caotic.

Why handling docs updates on a forum is a bad design choice?

  1. the conversation becomes very long and makes it hard to track changes (each new answer makes the subsequent patch at least l+1 harder, where l is the current length of the discussion)
  2. if two or more users submit a patch, then the answer of the maintainer is detached from the question (you can check the reply, but it’s not very intuitive, especially if the conversation on such issue has many questions and answers)
  3. if a patch has already been submitted, but not yet committed, someone coming to the page has hard time finding that, and would possibly duplicate the effort
  4. whenever you want to submit a patch, you need to make a vague description like “I mean the part where it says command X, Y, but the second time, not the first, the first is fine”
  5. the maintainer needs to figure where the patch applies and then do some copy/paste job, which is very error-prone
  6. the fact that all of the previous points make the work harder discourages contributing

My recommendation is to have the documentation being a collection of markdown files in a GitHub repo

Advantages of a docs.md GitHub repo:

  1. each conversation pertains to a single patch or issue
  2. because of 1, it scales horizontally, so patching has always the same difficulty regardless of the previous submissions
  3. when you send a patch, an automatic diff is generated and it’s clear where the issue lies and where your contribution is going
  4. the maintainer can eventually accept the contribution, if deemed acceptable, and it
  5. considering the audience of snap software, most if not all have already a GitHub account, and GH is their primary mean to contribute to some snap or to third party projects, so they need extra effort to contribute to a docs forum and register to a separate platform
  6. it’s already a common practice in many other projects, so embracing it has a much lower effort than changing method

An example of how the end result would look like

This is an example from a course I’m following, and to which I enjoyed contributing.
Website: https://fullstackopen.com/en
Example page: https://fullstackopen.com/en/part0/general_info
Same page on GH for contributions: https://github.com/fullstack-hy2020/fullstack-hy2020.github.io/edit/source/src/content/0/en/part0a.md

Note how all content pages have a Propose changes to material-button that links easily to the given page, similarly to how it works here with the docs.

The process of contributing to that project is very light and enjoyable, while contributing to snap and snapcraft on the contrary is much more painful, and there is no reason it needs to be like that

A shared frustration

Many people have risen such issues in the past, please, listen to your community (which can likely include clients), this is of mutual benefit.

P.S.

We are sharing our frustration because we really like the technology and the concept of snaps, and would likely this to succeed, but this becomes much harder if we cannot work properly

7 Likes
2

IIRC the original Snapcraft Docs site did based on GitHub-hosted Markdown files.

I personally have no complaints to the current system, though, as it really lowers the barrier of contributing to the documentation, disregarding the maintenance costs.

Fabio Colella via snapcraft.io forum@forum.snapcraft.io 於 2020年12月16日 週三 下午7:03 寫道：

3

Hi, I imagine that if it may have lowered the barrier, it was in the initial days, when the docs forum had very few messages, but now it becomes a very higher barrier to contributing.

Also, I’m not sure about how it lowered the barrier in first place. My assumption is that most people who read such docs are developers, most of which have already an account on Github or may want one to host their snap repo, so people that was going to use Github anyway.

With the current system, contributing depends on being registered on this forum, which I assume to have a much smaller audience than that of Github, even if we restrict the set only to snap users. Or is this wrong?

Can you make some counterexample of people who found this easier, and most importantly, that still find the current system easier not that messages are starting to pile up?

Personally, if I was a casual user, I would rather abandon the task than getting through such a long and confusing conversation :confused:

Edit: I also made a hopefully comprehensive list of the advantages of the Github system against the disadvantages of the current one, can you address any of those. I mean, in the case of the lower barrier, is it really so much lower and convenient, that we want to bear all those disadvantages?

4

This assumption might not be correct; what if someone wants to contribute but is specifically not a developer, so they can not contribute via code, but can do so by helping improve the documentation? This is a very common contribution scenario with FOSS projects.

While the assumption might be correct here, consider the above scenario. For such a contributor the “I need an account on (GitHub|snapcraft.io)” barrier is equivalent. The barrier might be even substantially lowered if/when the snapcraft forum implements single-sign-on support.

  • Daniel
1 Like
5

You are right, it could be useful in this regard to check if the current contributions came mainly from devs or from non-devs (we can e.g. assume non-dev all who have no Github or Gitlab account). Also, if one is not a developer, they could still submit an issue. In that case, many of the advantages I suggested do not hold, but it can still be very friendly on non-devs and keep each discussion in a separate thread. At the same time, it would make the job very very better to anyone else who can edit a markdown file and send a pull request (which is very straightforward when using the GH interface: when you edit a file of another repo, an automatic fork and branch is created, so you can easily send a pull request with that :blush: )

Again, I was mainly considering devs, so my assumption on this holds if it holds that most people interested in the snap / snapcraft docs are devs who have a Github account.

I get that indeed the barrier to an initial contribution by a non-dev to snap / snapcraft could possibly be made easier if you just need to write on a forum. But is this still true if the discussion becomes very long and hard to follow?

At the same time, lowering such barrier for non-devs has the price of poor maintainability, poor scalability and making the work for devs much more frustrating and complicated, is such price really worth? (And a consequence of such price can be a higher number of errors in the docs and fewer contributions)

To better take into account non-devs, we could possibly have two buttons at the bottom of each docs page:

  1. Edit this page (on GitHub)
  2. Report an issue on this page (on GitHub issues, or have a snapcrat.io page where the user logs with their Ubuntu account and they are allowed to fill a form that creates an issue on the GH docs project)

I get that having two options is not really the nicest thing, but I still see it as a huge step forward from the current solution which I think makes everybody unhappy

1 Like
6

I happen to agree with the OP, I personally think having the docs here is pretty clunky. As @Lin-Buo-Ren mentioned, our docs used to be hosted in github, and I loved that workflow. Changes could be reviewed and tweaked easily, and it was simple to have different versions of the docs for different versions of the project. Here it feels messy and impossible to properly coordinate.

One of my biggest complaints is documenting a new feature. When do I document it-- when the PR containing the feature lands? Oh that’s not contained in a release yet, I’ll hold off. In GitHub, I could draft the docs change at the same time as the feature, and that PR is ready to merge as soon as the feature is released. I can’t do that here, so by the time it’s finally contained in a release and matters to people, I’ve forgotten all about it.

That said, one of the problems with the previous approach was that the documentation was far divorced from the product being documented. It was trivially easy to forget about documentation completely as you landed new features if you weren’t used to that workflow. Given that we’re all on this forum all the time anyway this feels a little more… connected. I feel like that situation has improved here. I don’t think we’ve found the recipe for ultimate success, but there’s something to be said for trying new things.

We’re talking about updating docs, right? Not code. Docs held in github are still docs. You can even edit them in the browser these days, no git knowledge required. Now, I suppose the case could be made that markdown is a barrier.

4 Likes
7

I’m a bit confused by some of the terminology you use.

  • the conversation becomes very long and makes it hard to track changes (each new answer makes the subsequent patch at least l+1 harder, where l is the current length of the discussion)

In my opinion, comments should just be used to track suggestions and issues. If you want to change something, just edit the post.

  • if two or more users submit a patch, then the answer of the maintainer is detached from the question (you can check the reply, but it’s not very intuitive, especially if the conversation on such issue has many questions and answers)

I’m a bit surprised to see the terms “patch” and “maintainer” being used here. If you want to change something, just edit the post. I created a bunch of docs pages here but I expect anyone on this forum to edit the posts however they want.

My recommendation is to have the documentation being a collection of markdown files in a GitHub repo

I have a lot of experience with both methods and I much prefer contributing to documentation on this forum instead of on GitHub/Gitlab.

  • It makes drive-by contributions much easier. A GitHub PR is a much bigger commitment than simply editing a forum post.
  • It makes it much easier for people to give feedback on the docs. No need to create an issue, just write a one-liner underneath the docs itself. You can even see all the previous discussion in the same place.
  • It makes it much easier to find feedback. When I make a change to a page, I often read through the discussion to see if I can improve other small things.
  • It makes it much more satisfying to edit the docs. I get instant gratification when my changes are online after a few seconds. If I have an hour free to work on docs, I can do the entire workflow in that hour. I don’t need to follow-up at a later time to make sure my changes actually get merged etc. I feel empowered which makes me more inclined to spend my time here.
  • You don’t need to install anything, even if you want things to be perfect. The markdown on GitHub is fine for having a rough idea of what you are changing, but it’s never a perfect representation. Even with the github web editor, you need to run the test docs server in order to see how your changes will affect the actual docs pages. With this forum, I just change the post and refresh the page a few times until the changes are applied. Then I can see if everything looks the way I want it without having to install anything.

The snapcraft docs are in a much better state at the moment, and I think the forum is a big part of that.

That said, I do agree that the lack of a PR workflow is a downside to this approach but I think it’s a good trade-off. The advantages of having much easier community input in the docs far outweighs the downside of not having a PR workflow.

Many people have risen such issues in the past, please, listen to your community (which can likely include clients), this is of mutual benefit.

I think they listened to their community and thus switched to the forum. I myself am very happy with the move from github → the forum and I know many community members who share this idea. The number of complaints about the docs seems to have gradually gone down after the change, not up.

A final note: it’s important to think about who you’re optimizing for.

  • I think forum docs are optimized for occasional and new contributors. It’s very easy to rope people in, but once you get really into it, you start to see the cracks of this method.
  • GitHub docs are optimized for frequent and experienced contributors. It’s a bigger hurdle to get into it, but once you’re up to speed, you can go faster.
2 Likes
8

Honestly, I sincerely hope Snapcraft has so many people wanting to contribute to the documentation, including non-developers, that a message forum was the only way to handle the traffic.

My understanding of using a message forum as documentation was that it had less to do with ease of contribution to “official” documentation than it does with the success of sites like Stack Overflow, which often serve as the unofficial documentation for complex stuff like Kubernetes, Docker, and yes, Snapcraft. It seems like software that is complex and has a large enough user base tends to generate the same kind of “how-do-I” questions over and over again, which makes searching on them in a forum easy.

So, the logic would be, hey, why not store the docs in the same place as the messages? This way, users can search in one place, the same place all the experts hang out to answer questions. Just an educated guess on my part there. And I’m not advocating for it, just saying that I understand the logic even if I don’t quite agree it’s a great workflow for maintaining official product documentation.

1 Like
9

I discovered only now that editing the post was possible. And I imagine also other people missed this, as most of the comments are not for discussing or suggesting, but to really ask someone with edit rights to change something. If I make an edit, does it go immediately live or does someone review it before?

The advantage of a GH issue is that each issue has its own place. Instead, here it’s all together. Yes, you can read a very long conversation and easily get lost in it. On GitHub at least you know the conversation pertains to that specific issue and is worth reading.

Even keeping the current forum-approach , I think if we could have at least separate conversations for different topics, it would already improve things a lot. But at the moment they get all under the same conversations for each given doc. Perhaps sub-threading could help, but I’m not sure if this platform allows for anything like that.

That’s a tradeoff, you may know what you do, but someone, even me, might make a wrong edit and if can go live without anyone realising that for long :confused: Unfortunately for me it has been the opposite, I felt very satisfying to contribute to other projects which use a GH editing system, while I find the current one very confusing and frustrating, while I’m enthusiastic at what snap allows doing…

Indeed if you really want to check small details, this method is an advantage. In most cases, however, the markdown preview shows all that is needed.

Snapcraft got also much more stable and mature, so I’m not sure that’s entirely because of the switch

That’s what I meant exactly with the fact that’s not scalable :confused:

Considering the audience of the docs, especially about snapcraft, I think that’s a safe guess to consider how contributors likely are already using GH or GitLab and will likely host their snap on one of the two

10

I’m sorry, but I really have no idea why a non-developer would want to read and even contribute to snapcraft. With developers, I’m thinking about people who write code, but also packaging and so on. I don’t imagine a casual user ever willing to read it :thinking:

I agree with your view, and in fact, I think having such a forum in Q&A style is great and very very useful in general, I’m just suggesting not using it in this way for the docs :blush:

1 Like
11

I agree. I much prefer GitHub for storing official documentation. Better versioning, PRs, better workflow (IMO).

3 Likes