Channels 2.0 Implementation

[mvo]

Background

The snap store is providing a richer framework for channels now. This allow publishers to go beyond the current 4 channels we offer and provide custom channels.

The schema for the channel name is track/risk/branch – where track and branch are optional. If track is not given “latest” is assumed (track is also called series sometimes, we need to agree on terminology here, happy to follow guidance).

The relevant bugs are: lp:1675054 (and lp:1628640).

Potential issues

snap info output

The current suggestion for the UX for the new channels is to change the output of snap info from:

$ snap info etcd
name: etcd
channels:                     
  stable:     3.1.5      (46) 7MB  -

to

$ snap info etcd
channels:     
  - 2.3:
      candidate: 2.3.8 (39) 5MB -
      stable:    2.3.8 (22) 5MB -

This means the output will be incompatible with the previous output so we need to decide if we want to do this.

Approach The current implementation will change the output.

REST API

Related to the above is the question about the REST api. Currently we send the channel information as map[string]*ChannelSnapInfo. We could add a new field tracks and send a map[string]map[string]*ChannelSnapInfo with “track” -> “channel” -> info. Alternatively we could flatten the tracks information so that it becomes “$track/$channel” and we send in the existing REST “channels” map. E.g. {channels: {"2.3/stable": {"version": "2.3.8"}, "2.3/beta": {"version":"2.3.9-beta1"}}

Approach The current implementation will use a flattened channel map to be compatible with the existing clients. In addition it will add a sorted list of track names (sorting will be the same as we get from the server). The client can then sort by tracks.

Store is not providing a channel map for existing apps

Right now the store is not sending the new channel_maps_list for apps that don’t have custom channels (e.g. hello-world). This means we need to still build the fake channel map in the client. We need to clarify with the store if the plan is to send the map with “latest” for everyone.

Implementation

There is work in https://github.com/snapcore/snapd/pull/3141 to implement it.

[facundobatista]

After fixing a bug, we have channel_maps_list information back for all packages, including hello-world, see:

>>> pp(requests.get('https://search.apps.ubuntu.com/api/v1/snaps/details/hello-world?fields=name,revision,channel_maps_list', headers={'x-ubuntu-series': '16', 'x-ubuntu-architecture': 'all'}).json())
{'_links': {'curies': [{'href': 'https://wiki.ubuntu.com/AppStore/Interfaces/ClickPackageIndex#reltype_{rel}',
                        'name': 'clickindex',
                        'templated': True}],
            'self': {'href': 'https://search.apps.ubuntu.com/api/v1/snaps/details/hello-world?fields=name,revision,channel_maps_list'}},
 'channel_maps_list': [{'map': [{'binary_filesize': 20480,
                                 'channel': 'stable',
                                 'confinement': 'strict',
                                 'epoch': '0',
                                 'info': 'released',
                                 'revision': 27,
                                 'version': '6.3'},
                                {'binary_filesize': 20480,
                                 'channel': 'candidate',
                                 'confinement': 'strict',
                                 'epoch': '0',
                                 'info': 'released',
                                 'revision': 27,
                                 'version': '6.3'},
                                {'binary_filesize': 20480,
                                 'channel': 'beta',
                                 'confinement': 'strict',
                                 'epoch': '0',
                                 'info': 'released',
                                 'revision': 27,
                                 'version': '6.3'},
                                {'binary_filesize': 20480,
                                 'channel': 'edge',
                                 'confinement': 'strict',
                                 'epoch': '0',
                                 'info': 'released',
                                 'revision': 27,
                                 'version': '6.3'}],
                        'track': 'latest'}],
 'name': 'hello-world.canonical',
 'package_name': 'hello-world',
 'revision': 27}

[mvo]

Yay, thanks a lot! I removed the old fake-channels code now from my PR.

[niemeyer]

When we first discussed the changes coming into channels, one of the nice realizations was that we’d have to change very little on the snapd side, because channels are simply strings with no strong semantics associated with the names.

So, before we go on and interpret those fields, perhaps we should try to understand what we’re gaining from it.

In other words, note how the format of snap info may actually remain untouched:

Single “latest” track available:

$ snap info etcd
name: etcd
channels:                     
    stable:     3.1.5      (46) 7MB  -

Different track available:

$ snap info etcd
name: etcd
channels:                     
    2.3/stable:     3.1.5      (46) 7MB  -

Command line with “latest” track:

$ snap install --channel=stable mysnap

Command line with different track:

$ snap install --channel=2.3/stable mysnap

So do we really need to change anything in snapd, and if so what and why?

[mvo]

You are right @niemeyer - we can keep the info output untouched. There is a UX spec floating around that asked for the change, but I’m fine keeping the output as is. One thing I would like to do is ensure the sorting of the channels is the same as the sorting we get from the server side. My other question would be what to do with closed channels. Should we hide them? Or display them with a ^ (example below, this is what the UX spec suggests). But again, happy to return to the previous behaviour (which is to simply hide them).

Current output:

$ go build && ./snap info etcd hello
name:      etcd
summary:   "Resilient key-value store by CoreOS"
publisher: canonical
contact:   snappy-canonical-storeaccount@canonical.com
description: |
  Etcd is a high availability key-value store, implementing the RAFT
  algorithm to deal with failover within the etcd cluster.  Popular
  in the Docker community as a shared store of small but important
  data in a distributed application.
  
channels:                        
  - latest:                           
      stable:    3.1.5      (46) 7MB  -
      candidate: ^                    
      beta:      ^                    
      edge:      3.1+master (50) 10MB -
  - 2.3:                              
      stable:    2.3.8      (22) 5MB  -
      candidate: 2.3.8      (39) 5MB  -
      beta:      ^                    
      edge:      ^                    
  - 3.0:                              
      stable:    3.0.17     (24) 7MB  -
      candidate: 3.0.17     (40) 7MB  -
      beta:      ^                    
      edge:      ^                    
  - 3.1:                              
      stable:    3.1.5      (46) 7MB  -
      candidate: ^                    
      beta:      ^                    
      edge:      3.1+master (50) 10MB -
  - ingest:                           
      stable:    ingest-0.5 (12) 4kB  classic
      candidate: ^                    
      beta:      ^                    
      edge:      ^     

[chipaca]

In snap info, unless we group by track, it’s really messy output. @mvo has a pastebin of that somewhere. But I think that’s the only change needed: as per the PR, just add an ordered list of tracks to the /v2/snap/ json, and then snap info can very easily print things sorted and grouped accordingly, but no further change is needed snapd-side.

[chipaca]

Also note the ux in non-verbose mode is for a more compact layout, with risk levels layed out horizontally and one line per track. Really nice unless people go crazy with long version numbers. Something like

channels:   # stable           candidate    beta  edge
  latest:     3.1.5      (46)  ‹            ‹     3.1+master (50)
  2.3:        2.3.8      (22)  2.3.8  (39)  ‹     ‹
  3.0:        3.0.17     (24)  3.0.17 (40)  ‹     ‹
  3.1:        3.1.5      (46)  ‹            ‹     3.1+master (50)
  ingest:     ingest-0.5 (12)

[chipaca]

(this is not @mvo’s branch; I hope to get around to implementing this someday soon though)

[mvo]

Grouping by track is easy, it would look like:

name:      etcd
summary:   "Resilient key-value store by CoreOS"
publisher: canonical
contact:   snappy-canonical-storeaccount@canonical.com
description: |
  Etcd is a high availability key-value store, implementing the RAFT
  algorithm to deal with failover within the etcd cluster.  Popular
  in the Docker community as a shared store of small but important
  data in a distributed application.
  
channels:                           
  latest/stable:    3.1.5      (46) 7MB  -
  latest/candidate: ^                    
  latest/beta:      ^                    
  latest/edge:      3.1+master (50) 10MB -
  2.3/stable:       2.3.8      (22) 5MB  -
  2.3/candidate:    2.3.8      (39) 5MB  -
  2.3/beta:         ^                    
  2.3/edge:         ^                    
  3.0/stable:       3.0.17     (24) 7MB  -
  3.0/candidate:    3.0.17     (40) 7MB  -
  3.0/beta:         ^                    
  3.0/edge:         ^                    
  3.1/stable:       3.1.5      (46) 7MB  -
  3.1/candidate:    ^                    
  3.1/beta:         ^                    
  3.1/edge:         3.1+master (50) 10MB -
  ingest/stable:    ingest-0.5 (12) 4kB  classic
  ingest/candidate: ^                    
  ingest/beta:      ^                    
  ingest/edge:      ^                    

or if we omit the “^”:

name:      etcd
summary:   "Resilient key-value store by CoreOS"
publisher: canonical
contact:   snappy-canonical-storeaccount@canonical.com
description: |
  Etcd is a high availability key-value store, implementing the RAFT
  algorithm to deal with failover within the etcd cluster.  Popular
  in the Docker community as a shared store of small but important
  data in a distributed application.
  
channels:                        
  latest/stable: 3.1.5      (46) 7MB  -
  latest/edge:   3.1+master (50) 10MB -
  2.3/stable:    2.3.8      (22) 5MB  -
  2.3/candidate: 2.3.8      (39) 5MB  -
  3.0/stable:    3.0.17     (24) 7MB  -
  3.0/candidate: 3.0.17     (40) 7MB  -
  3.1/stable:    3.1.5      (46) 7MB  -
  3.1/edge:      3.1+master (50) 10MB -
  ingest/stable: ingest-0.5 (12) 4kB  classic

[chipaca]

by “grouping” I meant the two-level header of e.g. 2.3: {stable,candidate,beta,edge}

[niemeyer]

That last table looks quite nice. We can improve it a bit further, still without breaking the format, by spacing:

channels: 
  latest/stable: 3.1.5      (46) 7MB  -
  latest/edge:   3.1+master (50) 10MB -

  3.1/stable:    3.1.5      (46) 7MB  -
  3.1/edge:      3.1+master (50) 10MB -

  3.0/stable:    3.0.17     (24) 7MB  -
  3.0/candidate: 3.0.17     (40) 7MB  -

  2.3/stable:    2.3.8      (22) 5MB  -
  2.3/candidate: 2.3.8      (39) 5MB  -

  ingest/stable: ingest-0.5 (12) 4kB  classic

Besides the benefits mentioned above, this also means the table shows the same string that people would be using in the –channel flag, which is a relevant win.

The versions in the table should be ordered semantically using the typical algorithm. Even if that’s not always right, it’ll be better than a pure lexicographical sort.

@chipaca We shoudn’t transpose the matrix unless we do that in conjunction with snapcraft, otherwise it’ll be uncomfortable to be switching the brain back and forth between the two.

[chipaca]

oooh, hadn’t thought of this layout

[mvo]

The schema outlined in #11 by @niemeyer is now implemented in the linked PR.