It seems most times we change something in the snapcraft file, subsequent releases get rejected.
A manual review doesn’t happen automatically and has to be explicitly requested for each release. Even when that is done, nothing might happen until a post is made here. And even once the manual review is done and releases are unblocked, the next release may be rejected for the exact same reason as the previous one and the whole cycle begins again.
This is a very painful process, especially for small teams that have to deal with releases for multiple platforms/architectures/channels/stores every day. We simply can’t devote this much human time to getting our releases unblocked in the snap store.
The result is that users that use the regular package get updates immediately and users on the snap package might have to wait a week. If this was a todo list app, that probably wouldn’t be a big deal, but this is a browser. Sometimes we release fixes for zero days exploited in the wild and we need our users to get them ASAP.
For now, I’d like to request a manual review for our latest two releases (arm64 and x64) - stuck for a week on the same thing the previous two were stuck on.
But I’d also like to ask for some long-term solution to this.
Thanks for bringing this to our attention, I appreciate the professional way in which you’ve raised your frustrations.
We have a high volume of snaps that require review, and sometimes we aren’t able to get to every one of them as quickly as we would like. We are always looking for ways to improve both our tooling and the publisher experience. I cannot promise overnight change, but we will take your input on board. We try to optimise and automate the the most common failure cases, but of course there will always be corner cases.
For today, I have approved the 1.52.117 releases, however you will need to manually request review of the revisions uploaded yesterday (242 and 243) for version 1.52.122. No need to create a forum topic, just Request manual review in the dashboard and we will action it asap. This is a limitation of the store and review queue mechanics I’m afraid.
Just to add some more context here - looking at the most recent revisions of brave, it appears you have updated it to now include the setuid chrome-sandbox binary - inclusion of setuid binaries in snaps is a security risk and so these need to be manually approved, hence the time delay for a manual reviewer to be involved.
There is a mechanism to make this automatic by updating the review-tools package (which is used to do the automated scanning of snaps that are uploaded to the store) - this change has been made internally via https://code.launchpad.net/~0xnishit/review-tools/+git/review-tools/+merge/443492. To finish this process, a new version of review-tools needs to be deployed to the store dashboard - I tagged a new release today of review-tools and will coordinate with @roadmr to get this deployed to the store.
Blocking a release with newly suid binaries sounds sensible.
Blocking a chromium-based browser release for days, because the chrome-sandbox is suid - that’s questionable.
Doing the above, approving, then doing it again in the next release and blocking it for a week - that’s definitely a problem.
Do I understand correctly that our last release was rejected because of the missing review-tools change? I’m sure you’ve considered automating that step - seems like it could save publishers some pain.
Given that this was all about security, I’d like to suggest the introduction of a security review fast track that also wouldn’t require explicit requests for review, for packages that are security-sensitive, such as browsers. Blocking those releases might have serious consequences for the user, while blocking a release of, say, a calculator app probably wouldn’t.
@wknapik I would argue that a browser is a more security sensitive package than a calculator and so should go through a more thorough review, not less. I understand the frustration around the current process, but we need to balance various risks here. This change to allow brave to pass automated review with the setuid chrome-sandbox is being deployed within the next 24h so future uploads should not be blocked on this issue.
I’m not saying there should be less scrutiny. I’m saying there should be less delay in delivering the scrutiny.
That could be achieved for some packages by automatically requesting manual review when a release is rejected and putting the review at the front of the queue, rather than the back.
Some packages can spend longer in the review queue, while some should be reviewed ASAP. I’d put browsers in the latter category.
