Mapping snap store sections to XDG sections

In GNOME Software we currently have a set of hard-coded categories derived from the desktop file spec:

Clicking on these categories currently doesn’t show snaps, which is confusing to users and means snaps are less discoverable than .debs currently.

According to snapd we currently have the following “sections” which we can use when doing a search:

  • ops
  • database
  • internet-of-things
  • featured (special category used for promoting snaps - maps to “Editor’s Picks” in GNOME Software)
  • messaging
  • media

Editing the discovery details for a snap in dashboard.snapcraft.io we have the following “departments”:

  • Travel & Local
  • Graphics
  • Finance
  • Food & Drink
  • Books & Comics
  • Weather
  • Science & Engineering
  • Shopping
  • Universal Access/Accessibility
  • Music & Audio
  • Utilities
  • Social Networking
  • Productivity
  • News & Magazines
  • Entertainment
  • Developer Tools
  • Communication
  • Education
  • Lifestyle
  • Media & Video
  • Sports
  • Reference
  • Medical
  • Personalisation
  • Health & Fitness
  • Business

Open issues:

  • Are snap sections different to store categories? To make clients that can use this information we either need to expand the sections to cover all the departments or replace the departments in the store with appropriate sections.
  • Do we have a good mapping between sections / departments and the GNOME Software categories?
  • GNOME software expects to show all available applications when clicking on a category. This doesn’t make sense with the way snaps work, so we probably need to discuss with G-S upstream how best to show a search here instead.
  • The section / department information is not provided in snap.yaml, so installed snaps can’t be shown in a store section. Does it make sense for this information to be in snap.yaml? Can we cache the information from the store and use that?
  • Snaps that provide .desktop files have categories inside them. Is there a way we can make use of those? We only have .desktop file information for installed snaps, so it wouldn’t work for a search at the moment. We’d have to iterate over every installed snap to work out what categories are used which might be inefficient (needs a cache?).

Thoughts?

2 Likes

Hi Robert,

The “departments” appearing in the store are a leftover from the old click days, we are actively working on cleaning up the code to remove the click support, so this means that unless a different agreement is made in this topic, departments will die very soon (department is not required for snaps nor exposed in the APIs).

I agree the section information should be in some way present in the snap.yaml. If I recall correctly, when the section feature was designed, the intent was that the developer would define a “desired section” and then the store section admin would re-accommodate if necessary.

At the time @evan was quite involved with the design of the sections feature, so maybe he can help us answering the items above.

Natalia.

1 Like

+1 for this.
We launched a Snap yesterday, an email client, but the only way to find it right now is to search for Email.

1 Like

Natalia has is exactly right. The full spec is here.

Briefly, developers specify the set of sections they desire to be shown in. These do not come from a finite, hardcoded list of choices. Any hyphenated string can be used. However, we will only surface the 20 sections deemed most interesting to clients. Sections can map, so snaps with “internet-of-things” in their snap.yaml could appear in results for the “iot” section, and “datenbank” for “database.”

That link takes me to a sign-in page.

Asking for “any hyphenated string”, rather than “a finite, hardcoded list of choices”, is basically tagging. The issues with tagging are, in brief:

  • It may not occur to creators to add tags that would be useful if they were widely used, precisely because so far hardly anyone has used them.

  • Conversely, there may be a bandwagon effect where creators add a tag that is not interesting to readers, because many other creators have already used it, and/or because the tag doesn’t have a clear definition. (For example, “python” for an app that happens to be written in Python, vs. something that’s actually useful for Python development.)

  • As an extreme version of that, creators can spam tags (commonly seen with hashtags on Twitter, and tags on Tumblr).

  • There may be duelling versions of the same tag (for example, “colorpicker”, “color-picker”, “colour-picker”). Even if client software aggregates them, the split may mean that neither/none of them is popular enough to show up for developers in a list of suggested tags to use.

In Ubuntu Software Center (I’m to blame for the “departments” term, sorry!), we instead defined a list of categories and subcategories (with mappings from the crusty XDG categories, as well as from Debian “sections”). Any app could have a primary category+subcategory (which is where you’d be if you followed a direct link to the app), plus one secondary category+subcategory (to avoid spamming). Yes, defining and maintaining that list was work. But so was defining and maintaining all the rest of the UI design.

snapd still returns a very small set of sections:

$ curl -s --unix-socket /run/snapd.socket http://localhost/v2/sections|jq .
{
  "type": "sync",
  "status-code": 200,
  "status": "OK",
  "result": [
    "ops",
    "database",
    "internet-of-things",
    "featured",
    "messaging",
    "media"
  ]
}

Does anyone know the full list of active sections in the store? Is the reason there’s not a games section that snaps have not yet been tagged with this?

Perhaps we could generate some standard section names that we can map to the XDG section names so GNOME Software can make snaps more discoverable. Currently snaps are much harder to browse than .debs.

@noise will know the answer.

Sections has been a bit neglected as we pursued getting to a critical mass of snaps, but we now need to review the set and start bucketing appropriately. I’m coordinating with the advocacy team to get that done, but we may need to revisit the tagging vs. hierarchy discussion as well.

This is now done.

I don’t think we should limit ourselves to the XDG categories, which is a quite terse set of overly broad options. The expanded set is equally outmoded; I’m not sure how we’d classify something like Instagram.

Instagram could probably be categorised as Social although that does not appear in the XDG list. In this case it seems to lump these kinds of apps in “Network” which seems wrong.

I agree we shouldn’t limit ourselves to the XDG spec, however to fit in with existing stores it’s ideal to be able to map snaps into the existing categories. Any fully snap aware store can make better use of the sections (we’ll try and improve GNOME Software as much as possible but that’s the trade-off of using a common project).

Note that the categories are just the top-level XDG sections, the full set has more subcategories (though it’s definitely from the pre-social era). In the case of Instagram I’d expect it to show in both “Graphics and Photography” and “Communication & News” (which is where the Network items show).

Do the sections (e.g. ‘games’) contain:

  • all Snaps that are games
  • the first n Snaps that are games (ordered by priority?)
  • a set of featured Snaps that are games (as defined by the store admins?)

I think they should work how the current gnome software centre does them, i.e. each category includes all the packages which are associated to that category. So if a snap is categorised as a game then it will be included in the games category list, taking into account any pagination to reduce the list to a bite-sized chunk but still allowing discovery of all the packages. (For pagination see how a blog will show 10 most-recent/featured items and then put the previous ten in a second page, followed by a third for the next 10 again, and so-on…)

As soon as you restrict the list to “featured only” or another arbitrary filter you are potentially allowing for developers to be miffed that their snap isn’t on that list and thus is invisible to the public. Obviously this isn’t meant to restrict featuring, but more that the categories pages of the software centre are expected to contain everything in that category, not a curated list. If it were a curated list where would you expect to find the full list?

Upstream GNOME has changed behaviour to only show the top 50 highest rated apps in each category. This actually made it harder to integrate since snaps don’t have ratings and so were all filtered out :frowning:.

Note that snapd doesn’t support pagination when doing store searches - so searching for the ‘games’ section returns a finite number of snaps currently. So it might be possible to show all snaps in a category today but once the number of snaps in that section exceeds the threshold we’re already showing a subset.

An update on where we are with this (much progress was made at the Budapest sprint March 2018):

  • We map the current snapd sections to the toplevel XDG categories as featured apps (upstream and in 18.04). Snaps do not show in the subcategories.
  • We worked out an XDG subcategory mapping that the advocacy team will fill out with snaps.
  • Once we have the subcategories, we will update GNOME Software to show those.
  • We will backport the simplified categories in the version of GNOME Software in 18.04 to 16.04 (in progress). This will allow the snap subcategories to show the same in both 16.04 and 18.04.
2 Likes

Hey Robert

  • Is there an update on the progress of this?
  • can we get a complete set of categories/sub-categories being used by Gnome software? (if the set is different from the Free Desktop categories)

@evan

  • Could we get the same for what is being used by Snapcraft?
  • What other ‘verticals’ would we also like to support as categories? (IoT, Server etc)

I thought I had sent the sections but it seems I might not have. The sections GNOME Software uses named using a scheme that would work for snap sections are listed below. It is a subset of the XDG categories (some are grouped together into these categories). If you wanted to use XDG categories that would also work, but it would be more work to fill out.

featured
featured.banner

audio-video
audio-video.featured
audio-video.creation-editing
audio-video.music-players

developer-tools
developer-tools.featured
developer-tools.debuggers
developer-tools.ide

education-science
education-science.featured
education-science.artificial-intelligence
education-science.astronomy
education-science.chemistry
education-science.languages
education-science.math
education-science.robotics

games
games.featured
games.action
games.adventure
games.arcade
games.blocks
games.board
games.card
games.emulator
games.kids
games.logic
games.role-playing
games.sports
games.strategy

graphics
graphics.featured
graphics.3d
graphics.photography
graphics.scanning
graphics.vector
graphics.viewers

productivity
productivity.featured
productivity.calendar
productivity.database
productivity.finance
productivity.word-processor

addons.fonts
addons.codecs
addons.input-sources
addons.language-packs
addons.shell-extensions
addons.localization
addons.drivers

communication
communication.featured
communication.chat
communication.news
communication.web-browsers

utility
utility.featured
utility.text-editors

reference
reference.featured
reference.art
reference.biography
reference.comics
reference.fiction
reference.health
reference.history
reference.lifestyle
reference.politics
reference.sports

1 Like

@Greggless we started working on the ideal set of categories in Berlin, to be used in 18.10.

The set we have been using to date is:

  • developers
  • games
  • productivity
  • finance
  • graphics
  • video
  • music
  • social-networking
  • utilities

I’ll give the verticals some thought. I know @carlaberkers had ideas there as well.

1 Like