Search by common-id case sensitivity

kottens · June 20, 2024, 8:41am

While working with Discover and Plasma Welcome, I realized that the search by common-id on the search store endpoint is case sensitive. So for instance: curl -s "https://api.snapcraft.io/api/v1/snaps/search?common_id=org.videolan.VLC" and curl -s "https://api.snapcraft.io/api/v1/snaps/search?common_id=org.videolan.vlc" don’t give the same results.

This is unexpected because AFAICT other appstream related tooling treats this as case insensitive. For instance appstreamcli get org.videolan.vlc and appstreamcli get org.videolan.VLC give the same results.

Any chance to treat common_id as case insensitive on the search endpoint?

jamesh · July 12, 2024, 9:40am

As far as case sensitivity goes, the AppStream docs say:

Additionally, even though uppercase letters are permitted in a component-ID, it is strongly encouraged to only use lowercase letters for the ID.

Going by the appstream reference implementation, it is indeed case folding before doing a search:

github.com

ximion/appstream/blob/ce39fd2eb5ec3e701cf39be3ae372ca334629d05/src/as-cache.c#L1637-L1644


      
          	id_lower = g_utf8_strdown (id, -1);
          	xb_value_bindings_bind_str (xb_query_context_get_bindings (&context), 0, id_lower, NULL);
          	results = as_cache_query_components (cache,
          					     "components/component/id[lower-case(text())=?]/..",
          					     &context,
          					     0,
          					     FALSE,
          					     error);

Annoyingly, they’re doing locale dependent case folding. That would be difficult to fit into the existing store API, and hard to speed up with indexes. However, ASCII case folding would probably cover most of the cases people care about.

There is the issue that common-id is not specifically an AppStream ID. But it was originally added to expose an AppStream ID (e.g. this and this). I don’t think it has ever been used for any other purpose. So if ASCII case folding would help with the AppStream case, I think it seems like a good idea.

soumyaDghosh · July 12, 2024, 11:50am

@jamesh I was looking at the snapd code yesterday, and I think, we can write the desktop files based on the common id which can remove many theming and icon related issues in both KDE and Gnome.

https://github.com/snapcore/snapd-desktop-integration/pull/77

jamesh · July 15, 2024, 11:59am

That’s a bit of a different issue, and wouldn’t solve the store search issue being discussed here. I’d sketched out a proposal for this a while back here:

It’s something we’d like to implement, and there has been a bit more exploration since I made that post (e.g. it turns out gnome-shell makes some assumptions about snap desktop file names, which will need to be fixed). But we haven’t started on the implementation yet.

soumyaDghosh · July 15, 2024, 1:19pm

But, probably, there hasn’t been any more progress on that front. Because, that thread had stopped 4years ago.

EDIT: I shouldn’t spoil this thread though, as the topic is different. So, to confirm, I tested with several other ids and got the same thing.

kottens · July 22, 2024, 1:32pm

Yes, in practice ASCII case folding will be good enough in most case. I’m not aware of any AppStream ID containing non-ASCII characters.

Any chance to get the API to do this? Who would be able to modify the implementation for that purpose?

kottens · July 25, 2024, 11:22am

By the way I just realized that I was mentioning the v1 API, the point applies similarly to the v2 API. To be more precise this exhibits the same issue:
curl -s "https://api.snapcraft.io/v2/snaps/find?common-id=org.videolan.VLC -H 'user-agent: curl' -H 'snap-device-series: 16'
and
curl -s "https://api.snapcraft.io/v2/snaps/find?common-id=org.videolan.vlc -H 'user-agent: curl' -H 'snap-device-series: 16'
don’t give the same result.