Custom assertion possible?

om26er · May 2, 2021, 10:34pm

I want to install snaps from our S3 bucket without using the --dangerous flag. We’d sign the snap packages with our own private key and generate an assertion.

If it requires patching snapd, I can look into that as well.

Is something like that possible ?

futuretim · May 3, 2021, 8:00pm

You won’t be able to do this without patching for sure (afaik).

I have done a good portion of what’s required to make this work but it’s a tad complicated and you need a variety of assertions.

I’ve also started a store implementation but it’s only the most minimal to test some assumptions and start validating my ideas.

I am working on getting it in a state that is more ready to release as an open-source project but I’m not quite there yet. And even then it would be in just a “toy” state.

Just do a snap download on any snap and then look at the .assert file that comes with it and you can start to get an idea that it’s not quite so straight-forward as at least you described. Also, if you have interfaces that need to be connected those require assertions.

futuretim · May 3, 2021, 8:02pm

If you’re intent on pulling on this thread (like I was), I believe this is where you want to start:

https://github.com/snapcore/snapd/blob/5ab26a192094e0f6478a48ce698d703515888658/asserts/sysdb/generic.go#L30

And that will start you down a lot of different paths to understanding.

om26er · May 4, 2021, 9:57pm

Let me rephrase what we are looking to achieve. We want to only install snapd and core20 snaps from the public store. All other snaps will come from our own store. Snap interface connections will be dealt separately with our custom logic.

We want to add some sort of “trust” into snapd so it trusts our keys (I guess I am a bit off here in my understanding of things).

om26er · May 4, 2021, 10:00pm

I would be quite interested in that, maybe we could collaborate on that eventually (I am investigating currently on our available options. Exploring Brand Store as well)

ogra · May 5, 2021, 9:56am

you will need to loop your snaps at least once through a (brand)store to get a proper gpg signature (and the matching assertion) … then you do indeed not need the --dangerous option

i guess while you could hack snapd to accept additional keys, this is likely non-trivial to achieve… not to mention that you will need to quickly adapt your hack for each and every new snapd release to not have production devices run with an outdated snapd …

futuretim · May 5, 2021, 1:39pm

Just having snapd trust other accounts/account-keys is easy and not really hacky. In fact with Go 1.16+ and embed it could be very straight-forward to extract the default account/account-key assertions for the trust and have them as separate files. Then it wouldn’t be a hack at all to have a directory of “trusted” keys when you build. Or you could just load them on the fly from a separate location. There are reasons that all of this might not be desirable of course.

The non-trivial part comes in making all of that useful. That is definitely, 100% non-trivial.

futuretim · May 5, 2021, 1:43pm

That is the ultimate goal (to start collaborating with others). A brand store will be the most straight-forward and timely, if expensive, approach.

You would always have a patched/forked snapd with the other approach. That may be viable down the road but it definitely is not right now.

om26er · May 11, 2021, 10:23pm

One approach that I have been investigating is to implement a totally custom logic and deliver snap updates via a custom daemon (inspired by @ogra’s script in another thread) . And being able to generate a custom assertion is one important part of that.

Implement a way so that snaps uploaded to our “custom” store can be installed without using the --dangerous flag i.e. custom assertion
Write a daemon in python that
- Checks the currently installed snap packages and their versions
- Inquires the server if there is an update available
- Downloads and installs

Automatic interface connections could also be handled by that daemon. Of course there will be some server side logic for this as well

This kind of becomes the simplest approach out there, with much less engineering effort.

futuretim · May 12, 2021, 1:20pm

Going back to my original comment, did you inspect the contents of an assert file from a snap you downloaded? Try snap download hello-world and then look at hello-world.assert. You will need an assertion for the snap declaration and an assertion for each revision you want to load.

So you will need to patch snapd (to add your trusted key) and then you will need a way to push each new snap revision to your store and have it generate the declaration and then promote it / push it on the channel the snap is tracking on your device.

om26er · May 12, 2021, 11:45pm

I looked into the assert file in details and also went through https://snapcraft.io/docs/assertions. This does help understand quite a lot of things.

With effort, we should be able to implement something that generates the assert file on the server side, that we can pull on the host alongside a snap.

Implement a basic “account” system in the backend for (type: account assertion)
Allow that “account” to upload a snap and generate a type: snap-revision assertion from that
Need to do a similar treatment for type: snap-declaration

I assume this all isn’t going to be straightforward as we need to figure out how to create a signature for each assertion.

And of course this all depends on patching snapd to allow “our” account-key alongside canonical’s

At a later stage we could update our store backend API to implement all the endpoints required by snapd.

om26er · May 12, 2021, 11:49pm

So once we have this stuff working, the next step would be to deliver the snap with assert to the device and install it using an external daemon that talks to the snapd sock. This will allow the device to only install “trusted” packages and avoid the complexity of writing a full-fledged store with all the API endpoints that snapd calls.

Thoughts ?

futuretim · May 13, 2021, 2:03am

That seems possible. One thing that immediately comes to mind is refreshing snaps. I don’t know what the behavior would be of snapd if you were to do this and it went looking for information about these snaps.

You would effectively being doing a snap download, snap ack and then snap install as a side-channel with your trusted key.

Seem plausible.

Might be a place to start to answer those questions:

https://github.com/snapcore/snapd/blob/737be661ec6ad549602ad840d7986ed397778588/overlord/snapstate/autorefresh.go#L456

om26er · May 15, 2021, 10:19pm

So I have this stuff working now. Here is what I had to do

Add account and account-key assertions for our ORG in trusted.go and include them in trustedAssertions in that file.
created account, snap-declaration and snap-revision asserts by creating a json file for each and signing them with snap sign
create account-key assertion with a custom script inspired by https://gist.github.com/anonymouse64/f644d32f8fc24b98399cf2cb36891931 (still need to make effort to port this stuff over to Python)
Hacked snapd to not chicken out on “refresh” of snaps not available in store.

What I still haven’t figured out is how to generate the snap-sha3-384 of a file. running sha384sum produces a hex string, running openssl dgst -sha384 -binary < myassertion.snap | openssl enc -base64 also doesn’t produce desired result. Can anyone help with that please ?

I am sure there will be more roadblocks along the way, however this thing works in principle, which is exciting for me.

roadmr · May 16, 2021, 12:40am

Sha3 is a different algorithm (for example sha384sum doesn’t tell you this but it uses the sha2 algorithm). Find something that can handle sha3 (OpenSSL can with the right invocation - google it, or there are python libraries for sha3) and you might make a bit more progress.

om26er · May 16, 2021, 11:15am

Thanks, it worked. This is what I did in Python

import hashlib
import base64
import sys

def sha3_384(filename, blocksize=65536):
    hash = hashlib.sha3_384()
    with open(filename, "rb") as f:
        for block in iter(lambda: f.read(blocksize), b""):
            hash.update(block)

    return base64.urlsafe_b64encode(hash.digest())


print(sha3_384(sys.argv[1]))

om26er · May 17, 2021, 6:23pm

So now I am looking to create my own implementation of snap sign but written in Python. This should obviously be doable as I mostly need to find a way to sign the json string. However one thing I am having a bit of a hard time figuring is how is the sign-key-sha3-384 generated. Is that something to extract from the fingerprint ?

om26er@HomePC:~/work/src/github.com/snapcore/snapd$ gpg2  --homedir /home/om26er/.snap/gnupg/ --list-keys
/home/om26er/.snap/gnupg/pubring.gpg
------------------------------------
pub   rsa4096 2016-01-01 [SCEA]
      71D820A8945C231AD5948F529DADA9FEC015A12F
uid           [ultimate] omer

Here is what I tried

>>> import binascii
>>> import base64
>>> import hashlib
>>> raw = binascii.a2b_hex("71D820A8945C231AD5948F529DADA9FEC015A12F")
>>> hash = hashlib.sha3_384()
>>> hash.update(raw)
>>> print(base64.urlsafe_b64encode(hash.digest()))
b'w8kg8wjjFGKq11wqkl7GfIUN8F-_4hAUgk6vyB2CpTiNU9Mx90b0t3Ewwlv42Q81'


om26er@HomePC:~/work/src/github.com/snapcore/snapd$ snap keys
Name     SHA3-384
omer     IjxSYf7DvHlK6ArTRvPRfj4vDjSk0ughEnLWm9nZLbnJCvmXuNwUsnRk2-1g2agL

And the output is clearly different

om26er · May 17, 2021, 7:34pm

Ok, found a way. I had to read the base64 of the public key, decode that (to get bytes) and then pass to the hash function.

om26er · May 18, 2021, 3:01pm

So the last missing piece that I have been struggling with is to port signing of the document. I am trying to use gpg cli tool to sign the document that looks like this

type: account
authority-id: 4d6cd738e4914c50b71c87268b0cdd2c
account-id: 4d6cd738e4914c50b71c87268b0cdd2d
display-name: Omer Akram
timestamp: 2021-05-15T10:23:51+00:00
username: om26er
validation: unproven
sign-key-sha3-384: Dw8e6E_xmIKKonaOWDiGK_G9kwhCRRqhaMx-t3qV8-hraLQlGuyw-QcqSbr-aoWv

However the output result is “different” and snap ack doesn’t accept it. Am I missing something or is there something special that I need to take care of ?

I have also attached the sample private key

Could anyone help with that please ?

PS: While I am just getting started with the snapd code (a week or so), my initial impression is that it’s quite complicated code to “understand” and lacks comments at many places

futuretim · May 18, 2021, 4:45pm

Do you have open-source repos were contributors could help?