ISV questions about signing a snap

We are working with an ISV who create a browser, while reviewing their snap process they realized their snaps aren’t signed before they’re submitted to the store. They have seen that the following commands are available even though they’re not mentioned in https://docs.snapcraft.io/

  • snapcraft sign-build
  • snapcraft create-key
  • snapcraft register-key
  • snapcraft list-keys

They have the following questions about them:

  • Is it possible to extract a signing key to store/import it elsewhere?
  • Once a key is registered, does the store refuses unsigned snaps?
  • Can we rotate theses keys? More precisely, can we register a new key and make the old one invalid?
  • Are the signatures checked by the snapd clients, or only on the store?
  • What are “assertions” in the context of snaps?

I’m posting their questions here so the answers gathered might be of assistance to others.

@cprov Can you provide some answers?

1 Like

Before we dive into the specific question, let me clarify your initial statement. Snap are signed and snapd checks it before installation. This process assures the snap about to be installed is the one uploaded, reviewed and assigned with a unique revision by the store. The trust-path is established cryptographically between snapd and the Store, i.e. the snap is signed by the Store, derived from authenticated, secure and auditable uploads from snapcraft.

That said, there are also mechanisms in place for allowing accounts (users) to sign specific contents when it’s necessary or desirable. These primitives are already used for creating devices (models), validate snaps and optionally sign builds (revisions).

Is it possible to extract a signing key to store/import it elsewhere?

Yes, create/register/list/export mechanisms are in place. However their usefulness depend heavily on which process you have in mind.

Once a key is registered, does the store refuses unsigned snaps?

Snaps are always signed, it’s not optional. They can be optionally signed by the publisher and we have planned work to also optionally check publisher signature before installation (obs: in practical terms, it would be just checking a further link in the trust chain, since the publisher key is also trusted via the Store key)

Can we rotate theses keys? More precisely, can we register a new key and make the old one invalid?

You can register new keys right now (commands you identified) but revocation is done via manual request if and when they get compromised. Work on complete key-management features is planned but not yet prioritized.

Are the signatures checked by the snapd clients, or only on the store?

Snapd clients check them before install or refresh.

What are “assertions” in the context of snaps?

Assertions are the internal documents (cryptographically signed) representing all the global aspects of snaps trusted by the whole ecosystem (client, server, 3rd-parties). For instance, the snap_id of a given name, or given a hash/digest of a snap blob reviewed by the store, what is the (snap_id, revision) and so on. While crucial to snaps security, assertions are rarely exposed in day-to-day operations, they are meant to be processed by machines :wink:

I hope the answers above clarified your points and the SnapStore and Snapd teams are available to discuss requirements with any ISV interested in directly (re-)signing snaps with their own keys and potentially establishing additional (out-of-band) security.

3 Likes

To clarify, snapd is checking the snap-revision assertion but not currently snap-build. More info on the various assertions can be found at https://docs.ubuntu.com/core/en/reference/assertions

1 Like

To clarify the clarification, :slight_smile: both of those assertions do include the digest, and are properly signed. So the device does check that the digest matches the one uploaded.

1 Like

the current export is an internal mechanism only for the public part, there’s no official import/export atm for secret keys, though the actual keyring in the end is GPG so it’s doable but not yet supported through proper commands

Thank you all for your precise and rapid answers :smiley: I’m the person who asked the original questions. I appreciate the details on what’s happening on the client side. That helps to get the bigger picture. If you don’t mind, I now would like to focus on what happens when a Snap is uploaded onto the store. I’d like to clarify a few more things. I left the details below.

I’m sorry, I’m a bit confused by these 2 statements. Where does the Store signature happen? Is it on on the store itself, after the snap is uploaded? Or is it on the machine that uploads the snap, right before the snap is uploaded? (I guess the former is correct, because the upload machine shouldn’t know about the Store private key, but maybe there’s a more complex mechanism that I missed)

Our main goal is to sign Snaps before they are uploaded. The main reason is: we want to have 2 factors to validate the origin of the Snap:

  • The macaroon used to authenticate to the store
  • The publisher signature of the snap itself.

That way, an attacker needs to comprise these 2 factors if they want to publish a rogue Snap. If I understand the current situation, getting a copy of the macaroon is enough to get something uploaded. Does that sound about right?

I’m sorry, I’m not sure to follow. What happens if I sign a Snap with a revoked publisher key? Does the store prevent the upload? Or will it accept it, until the feature is implemented?

We would like to backup the publisher private key, in case we lost the machine the key lived on. Great to hear it’s GPG! It’s gonna be straightforward to backup. Where does the keyring live after I ran snapcraft create-key?

Makes sense. We’re likely going to rotate keys once every year or so. Contacting a human works for us.

I see, the idea is to increase security on the upload path. The snapcraft default macaroon is very powerful in this workflow, it not only allows the principal to push and release new revisions, but also allows creation of new keys (!!!). This means one possessing the macaroon can easily trumps any attempt to check upload-specific signatures by creating a new key and signing the upload, it is just a further link in the same trust chain.

I think we can solve this problem in a much more effective way by exploring the macaroon attenuation mechanisms we already support in the Store. Instead of trying to circumvent a super-powerful macaroon during its lifecycle, we could allow them to be issued in a more strict scope, e.g. only for pushing uploads (not creating keys), of a specific snap, to a specific channel (edge) and with a very short expiration (few hours). Issuing new macaroons requires SSO authentication, which already supports 2FA. Snapcraft login “sessions” could have a very narrow scope and timespan when desired.

Snapcraft already does something along these lines for enabling 3rd-party CI services, e.g. snapcraft enable-ci travis and we can design UX changes to make it possible to ‘snapcraft login’ in a very restricted scenario, whereas the macaroon that gets compromised will be most certainly useless by the time it can get exploited.

Would that address your concerns ?

Upload and signing are currently decoupled, the upload process is effective based on macaroon-auth & SSL and if later attempted to be signed with a revoked key, only the signature is refused.

It lives in ~/.snap/gnupg/

Oops, this statement is not accurate, snapcraft already forces users to re-autheticate (SSO password + 2FA) in order to add a new key and it is immediately discarded. The default snapcraft macaroon is not as powerful as I painted :wink:

My original point remains, though. We can prevent damage by scoping sessions natively and it would be more effective than additional circumventions.

Understood!

I think restricting the scopes of the macaroon is definitely a good idea (even if the default macaroon is not that powerful). We’ve already set ours to push to the edge channel only. Regarding short expiration, I believe that works when releases are not too frequent. A human generates the macaroon used for the day and repeats the operation at every release.

In our case, we currently upload a new snap twice a week. We may eventually ship twice a day, thanks to our automation. Based on that, we need macaroons that live long enough so that a person doesn’t have to bother about them on regular basis.

But then, long-lived macaroons are very harmful if they leak. Therefore, I’d more comfortable if there was another way to enforce the origin of an uploaded Snap. Checking the publisher signature on the Store side is one way. If you see another, I would love to hear it :slight_smile:

Yes, I can understand people are more comfortable automating GPG signatures than macaroons, it’s probably already part of most CI systems they already have, although there will be nuances supporting snapcraft sign-builds <snap> as is. Also, there is a slightly improve in security by associating both, scoped macaroons and upload signatures; an attacker would have to compromise both vectors to succeed, as you mentioned before.

I don’t see any technical blocker for optionally requiring corresponding snap-build assertions (output of sign-build) to be uploaded along new snap revisions. Additionally to all the existing macaroon authentication, the Store would check if the revision matches the signature before carry on with the usual upload processing (snapcraft push <snap> would automatically pick a <snap>.snap-build, if there is one).

Let’s wait a bit to see if other people in this thread have any objections for supporting this ‘hardened’ upload workflow and thanks for clarifying your points.

Yeah this would be super useful. I also like the idea of the “hardened” upload approach; it feels familiar.

@cprov Sounds reasonable to me as well. Let’s please not have a .snap-build, though. We currently have a .assert when we download a snap, with the same name of the snap. Along those lines, having a .build next to a .snap seems more consistent.

@niemeyer Good point, .build is more consistent and cleaner (no dash). I will propose the fix in snapcraft. Thank you.

Proposed changes to snapcraft use .build extension: https://github.com/snapcore/snapcraft/pull/1757

I missed these last bit discussion, fwiw the pattern that was followed before was fname.<assertion-type> , which we use in a few places (snap prepare uses it for example)

As mentioned earlier, snapcraft and the Store now support more granular macaroon attenuations (along with short expiration). See more details in Better support for other CI systems

We haven’t prioritized the hardened upload task yet. Please, let us know of any specific use-case blocked on this.

Thank you for making macaroons safer to use! That will be a good opportunity for us to give our CI package_push and package_release.

That said, if those specific creds are leaked out (either because of a human mistake or because an attacker managed to get some CI privileges), then anybody can still push/release a snap that doesn’t even need to be our product (as long as the snap metadata are correct). If signatures are enforced on the store side, then the attacker has to also get access to the private signing key.

We are not blocked on this, per se. We are also making sure to store and use macaroons as securely as possible. However, I think any open-source project that has CI logs public would benefit from signature enforcement. Inadvertently making credentials public (e.g. by logging out, or by committing them) happens. For instance, they are bots which monitor GitHub for leaked credentials (this used to be doable via a regular search).

@cprov I have a follow up question: How does one revoke a macaroon if it ever leaks?

@JohanLorenzo in case of a macaroon leak (exported login file or the Authorization header), you should immediately change the SSO account password, this will limit the validity of the leaked macaroon in 24 h (since it’s last auto-refresh). Also contact the store team (store tag in the forum is fine) for additional restrictions and general account auditing.

We reviewed this assessment. To us, it doesn’t sound acceptable to leave a 24h-window (not including the time we discover it) where attackers can do anything they want with our brand. For the context, we patch/build/test/ship in-product security fixes faster than that.

This window would be highly mitigated if signatures were checked, meaning an attacker has to steal 2 things (instead of 1) to ship a rogue browser.

Like said in post 17, this is not blocker, but I’d like to formally say this is high priority to us.