This means that if in the Store we receive a package with the “epoch read” field unordered, we should reject it?
Let’s think about this scenario:
| revision | epoch |
|---|---|
| 3 | 0 |
| 6 | 1** |
| 8 | 1 |
With the previous way of thinking, a transition from rev 3 to 6 would have been ok, and a transition from 6 to 8 would also be ok (these are cases 4.3 and 4.7 in the spreadsheet)
However, with the new syntax, the above scenarios translates to:
| revision | reads | writes |
|---|---|---|
| 3 | [0] | 0 |
| 6 | [1] | 0 |
| 8 | [1] | 1 |
Here, we see that
-
can not jump from 3 to 6, as 3 writes e0 which is not readable by 6
-
can not jump from 6 to 8, as 6 writes e0 which is not readable by 8
Beyond those fixes, I worked on Scenario 1, where I simplified some cases (with this new “read/write capabilities” several cases were just explicitly redundant), and added a case that wasn’t able to be expressed before.
Also, for all scenarios I translated the old syntax to the new one, for better reasoning, and changed the cases comments to acknowledge the new rules.
Finally, I highlighted in red some cases that conflicts with previous rules (which motivated my comment/question in the previous post).
All that said, I kept a “backup” tab in the spreadsheet with the “old way” cases.
Yes, as that’d be a broken snap. This should have been rejected by snapcraft and snapd before.
Okay, but as I said above I won’t touch the Scenario 1 other than fixing obvious typos, because that would make the conversation confusing since we’d have two Scenario 1s. I suggest taking changes into a new scenario instead (numbers are cheap 
). Also, it’s my understanding that pretty much everything in the old scenarios should remain working, so it does seem useful to not corrupt them and instead just add new cases that are relevant for the modifications introduced.
You are right… ironically the “full clarity” provided an incorrect example. Just like 1*, 1** should also include a read of 0. I’ll fix the example and note the update there.
Actually, there’s a trivial decision to make about which order we should enforce in the field. Perhaps something like [3, 2, 1] reads more clearly since the 3 here is the highest one and will usually be the most relevant for the application version at hand, and also plays an important role in the refresh process.
Reverted then to original redundant examples (but kept the better comments).
Also created a new scenario at the end for some of the new combinations.
1 Like
Still not working, see (repeating some of the previous post just for readability):
| revision | epoch |
|---|---|
| 3 | 0 |
| 6 | 1** |
| 8 | 1 |
With the previous way of thinking, a transition from rev 3 to 6 would have been ok, and a transition from 6 to 8 would also be ok (these are cases 4.3 and 4.7 in the spreadsheet)
However, with the new syntax, including your last change, the above scenarios translates to:
| revision | reads | writes |
|---|---|---|
| 3 | [0] | 0 |
| 6 | [0, 1] | 0 |
| 8 | [1] | 1 |
Here, we see that
-
jumping from 3 to 6 is ok, as 3 writes e0 which is readable by 6
-
can not jump from 6 to 8, as 6 writes e0 which is not readable by 8
1 Like
You are right. Thinking through it…
Hello!
Do you have any news on this? If we nail down this last issue we at least can confirm that the “new syntax” is solid and doesn’t have structural problems (which may led us to going back to the old one).
Thanks!
Apologies for the delay. I’ve been trying to come up with a better idea that would avoid the complexity of the explicit syntax, but I’m finally appreciating the explicit syntax as it’s completely clear about the underlying mechanism, and it’s only supposed to be used rarely since we have a nice notation covering the typical case.
So here is the updated proposal which simply tweaks the one above with the updated write notation, pretty close to what was originally suggested by @wgrant I believe:
| Example | Expression | Expanded | |
|---|---|---|---|
| A | (missing) | epoch: read: [0] write: [0] |
|
| B | epoch: 1 | epoch: read: [1] write: [1] |
|
| C | epoch: 2* | epoch: read: [1, 2] write: [2] |
|
| D | epoch: write: [1, 2] |
epoch: read: [1, 2] write: [1, 2] |
|
| E | epoch: read: [1, 2] |
epoch: read: [1, 2] write: [2] |
|
| F | epoch: read: [1, 2, 3] write: [1] |
epoch: read: [1, 2, 3] write: [1] |
|
| G | # Syntax not supported! # |
epoch: read: [1, 2] write: [1, 2] |
The proposed rules are:
- If any of the the write epochs of the snap currently installed is present in the read epoch list of the candidate snap, snapd will accept the refresh.
- The store always offers the most recent snap able to read the highest epoch among those that may be installed (see rule 1 and 6).
- The default write epoch list, when one is not provided explicitly, contains a single epoch which is the highest one in the read epoch list. That makes sense as software generally reads old formats but writes in the new format.
- The default read epoch list, when one is not provided explicitly, is a copy of the write epoch list. That makes sense as software generally can read what it writes itself.
- For clarity, the read and write fields must be ordered when provided explicitly in the yaml file, which means the highest epoch is always the last one in the list.
- When a snap is pushed to the store it becomes the most recent and best candidate on all epoch reads it supports (addresses point discussed on case 3.4 above).
In a more prosaic form:
Rule 1 states the obvious: we can only install something that can read the data currently stored.
Rule 2 preserves the mental model relatively simple by having a single governing principle on refreshes: the goal at any one point is reaching the highest read epoch. Then, of course, among the available snaps offering that epoch, the most recently released one wins.
Rules 3, 4, and 5 are just about parsing of the extended syntax.
Then rule 6 clarifies what “most recently released” means and ensures that a snap released will only be more recent if the epoch being looked for is present in its read list.
How does that sound?
It sounds great!
And it feels solid. I adapted the spreadsheet cases to this new “write is also a list” concept, and all is ok now. Just in case, please review the more “crazy” combinations, in Scenario 6, but I think we’re good.
Thanks!!
We’re now firm on the “read/write capabilities”, not in the old “backward/forward transition”, so we need to rethink the proposed names.
Every release has now “a list of epochs that is capable of reading” and “a list of epochs that is capable of writing”.
Just to start rolling, I propose “epochs_reads” and “epochs_writes”
- the S in the readS/writeS is because "the release readS/writeS those epochs
- the S in epochS is because it’s a list
- maybe better names are reads_epochs and write_epochs, but I like it more the way around just to have them “alphabetically closer”
@facundobatista Not sure about what’s the context for this, but my proposal is still the one above: with the nested map instead of top level keys, and using the terminology epoch read list and epoch write list.
I’d definitely avoid “reads” and “writes” in this context, as you’ll want to use those terms as nouns, which means those would become plurals as in “five reads” instead of verbs as in “the snap reads”. Let alone “epochs_reads”… that sounds like a pain to code and talk about.
We may have a problem: see scenario 4 in the spreadsheet.
Those are cases triggered by the client requesting a specific revision, and the server deciding if it’s ok to let the client refresh by applying to the epoch rules.
When a user does a snap refresh NAME --revision=N, snapd hits this endpoint: /api/v1/snaps/details/.
The problem is that the information sent to the server in this case does not include the revision (nor the epoch) in which the snap is in the client, so we don’t have enough information server side to decide.
What can we do in this case? Thanks!
we likely need to reconsider what we use on install anyway because /details is a GET and sending lots of info to that seems not ideal (like we need to send all installed snap-ids because of last-validated needs)
Note that this issue happens on a refresh when specifying --revision, not on install (the Stepped Updates concept is not applied at install time)
yes, but /details is the same endpoint as install
real refresh use /metadata
we could also do the check only on the client in that case, then we just need epoch info in /details
Epoch information is in the /details response already (however, we’re sending the old-just-integer value; in the future we’ll be sending the pair of read/write integers lists.
In any case, if you would need to change snapd to process the epochs and do the right thing, why not change snapd to start sending the currently installed revision (if it’s a refresh!), and let’s just implement all the rules server side?
we need to implement the rules in the client too for safety, but as I said I’m happy to send more context, /details is probably the wrong endpoint for that though
1 Like