Adressing the low-quality snaps issue


#1

The snap store made it very easy to publish applications straight to Ubuntu desktop users. This has the unintended side-effect that it contains many low-quality and even broken applications.

I think that the snap store is in real danger of getting the reputation as “being full of broken apps”. This also impacts people’s view on how stable and useful Ubuntu is.

Are there any plans to address this issue? I’d like to have a free-form discussion first and see what’s already being done before I start filing bug reports etc.

Some ideas:

  • Making it possible for the community to help curate “editor’s picks”. For example, have a place to report broken “editor’s picks” applications such as Vectr.
  • Surface reviews to the publishers. Currently publishers have to install Ubuntu and open the software center to see the feedback of users. https://github.com/canonical-web-and-design/snapcraft.io/issues/2248
  • Have a way for publishers to remove old snaps. The minecraft-nsg snap, for example, is apparently abandoned, but it still shows up in the software center.
  • Make the search algorithm smarter. Look further down below for an example. Adding keywords to the applications would be a great way to improve search results.

Search result example

Let’s say a user is looking for a more advanced screenshot tool. This is what Ubuntu Software offers them. (click on the image for much more results).

I annotated which are broken, irrelevant, and how many 5-star ratings each has. There is so much wrong in this picture.

  1. Why are so many broken apps right on top?
  2. Why are there so many irrelevant apps?
  3. Why are screencloud and winsnap above the spectacle snap? They’re all snaps and spectacle has much better and much more ratings.
  4. Why are all snaps strictly above .deb packages, even if the snaps are reported broken by almost all reviewers?
  5. Why is “peruse” even in there since the only available channel is candidate?


#2

I would love to see the ratings and reviews system and processes around it expanded. Eg, perhaps add an optional section like:

Rating
   ☆ ☆ ☆ ☆ ☆

Summary
- Give a short summary of your review...


Review:
- What do you think of the app...


(Optional) This application:
☐ has a security issue
☑ is non-functional

(this last part would need design of course)

The snapcrafters team could then monitor anonymized data for snaps with these boxes checked and act accordingly. Today we encourage people to contact upstream, but that isn’t always easy for users to find where to do that, and there are times that the store/snapcrafters teams should be alerted to issues.

Furthermore, I’m not sure if this is still an issue, but AIUI, (at some point at least) ratings and reviews were not editable after they were submitted. If that is still the case, that should be fixed.

cc @advocacy


#3

Note: I just tested it in 19.04 and it seems reviews are still not editable.


#4

The ratings and reviews system is actually submitting data to, and querying ODRS. I think @robert.ancell can probably speak to that topic a little.

I would certainly like to see an improved way for applications which are known good, working and with decent reviews to bubble up the list of search results. Those which have low reviews should naturally get pushed down. Indeed, the fact that Vectr in this instance did have a bunch of 1 star reviews is exactly what users should use to help inform their decision about whether to install the app or not. I know I certainly use that as a measure on the play store.


#5

How do you think this would impact the issue of low-quality snaps?


#6

Perhaps if someone gave a great review and then wanted update it to alert others to a problem. I was more thinking the other way: someone reports that the snap is broken and then wants to say it is all better once there is an update.


#7

If you have ideas / issues with the review system you can report those against the ODRS project. I wonder if the sort of idea that @jdstrand is suggesting might be more of a “report a problem” type feature rather than being in the review system. The reviews UI is designed to be fairly simple as users tend not to fill out complex forms with much accuracy.

The order of search results comes from the Snap Store (i.e. the website), and this doesn’t currently have any information on review data. GNOME Software overlays the reviews onto the results from snapd. For us to filter highly reviewed snaps to the top the store would need to query ODRS. @noise - is that something that would make sense?


#8

Thanks for the response, Robert!

This seems like a discussion the Gnome folks might have some opinions on. Do you think it’s a good idea to continue this discussion in the ODRS project issue tracker? It seems something that reaches beyond simply snapcraft.io. Does Gnome have a less-structured way for discussing these types of things (that might not be specific to one project)

Ok, this sheds some light on the issue. Some more questions about this:

  • How does Gnome Software do search results for Flatpaks + rpms/debs/…?
  • Why is search implemented server-side instead of client-side?
  • Is the code that implements the search functionality open source?

#9

A question specifically for the store team (@roadmr ?):

Where are things like featured and category listings saved? Is there an official process (for the community) to request adding/removing apps?

For example, say I want to get a bunch of volunteers together to check every single featured application to see if the app meets some basic quality standards (like not segfaulting on start). Is there a way for us to get a list of all the featured applications?


#10

Hi! This is more of a question for @popey - technically the listings are stored in a database in the snap store infrastructure but the Advocacy team (popey’s crew) are the ones that curate the listings.

  • Daniel

#11

You could poke the API to get the current list of featured applications. They’re just sections so you can snap find --section to list the sections and then snap find --section server-and-cloud for example.

If there are issues with the snaps you fine, my recommended first port of call is to contact the publisher, either their issue tracker or via whatever means they list on their store page.

In addition, if there are applications which the community feel should be highlighted, let us know, and we’ll consider it for the next update.


#12

I would think the issue tracker is a good place to start. ODRS is done by one developer (Richard Hughes), so you could also ping him on IRC (nick hughsie) if you wanted to discuss first.

ODRS is used by a number of desktops, so any new functionality there would need to work for not just snaps.


#13

Do the “featured banner” apps in Ubuntu Software come from the same source?


#15

Yes, featured apps in Ubuntu Software, and the list on snapcraft.io/store are from the same place.


#16

I very much support this topic, especially the so many irrelevant apps that come up from search and that snaps are strictly above deb packages. I almost always ignore the snap results that come up first if there is a deb alternative in the archive.


#17

That doesn’t seem so easy. For example, search for “Blender” gets you:-


(Note that “blender-tpaw” has no reviews/ratings).

Now click through on “blender-tpaw” to see the page:-

It now has 5 stars in the top right, and unless I am being daft, absolutely no indication of how you go about contacting the person responsible for uploading the snap.

I find the snap store / ubuntu software a very confusing place to be. More often than not I fall back to doing a google search and installing from a different source.


#18

Search results are done in parallel and then combined together. This is a problem as there’s not an easy method to interleave the results i.e. how do you know a search for a particular snap is better / worse that a deb?

You’d need to ask a store person why the searches are done server side, but it helps that clients don’t have to download a large amount of metadata and keep it synchronised.

The search functionality is in the store implementation which is closed source (sorry for the delay replying, I was confirming this).


#19

That would certainly work for me. Is there a standard around this like there is for ratings and reviews? @noise and @advocacy - what would be involved in such a feature?