We’re encountering a situation where we need our snap to both set and get configurations. However, we require a central storage system for configurations that is shared across multiple nodes. We’re comfortable managing this shared storage ourselves (using dqlite, in our case), but we need the ability to return our specified values instead of the default behavior of snapd.
For example, if a user sets the configuration option option1=true on node 1, then calling snap get option1 on node 2 should return true.
Currently, snapd stores configurations locally. While intercepting the set command with a configure hook is straightforward, we’re facing an issue with the get command. It retrieves the value directly without invoking any hooks, thus not allowing administrators to influence the returned value.
Is there an undocumented hook or any other method to override the snapctl get command with a custom one? Alternatively, do you have a solution for achieving cross-node configurations?
A quick peek at the code (overlord/hookstate/ctlcmd/get.go) suggests that there is no ability for a snap to override this. The intent is, I believe, that snapd is the storage of state while allowing the snap to manipulate that. This means that unless you call snapctl get, it does not matter that snapd has the state. You can update the snapd-managed state at any time.
If I were you, I would implement the configure hook so that locally made changes are propagated to other nodes and incoming state changes (apologies if my understanding of dqlite is simplistic) is reflected back to a local instance of snapd.
Consider this situation: Suppose the user has installed myapp on two hosts. What event or action can the instances share? Given that updating the value in dqlite is straightforward, how can we inform the snap on host2 about an update in dqlite and prompt it to execute snapctl set to update local settings?
Users don’t run snapctl. Users can only use snap {get,set}. The snapctl program is meant for consumption inside the snap, from hooks or in general.
Apart from that it looks plausible.
A snap can run snapctl at any time. There is some special behavior when invoked from a configure hook, as the snap interacts with the in-progress transaction at that time.
Looks like we may have left a feature behind when working on the configuration. I just had a call with Samuele and Miguel on a related topic and they’ll look into this, but basically when we designed the configuration system in snapd we always had in mind that snapd would not be the authority for configuration data in many cases. For example, we may want to have snapd being responsible for a textual configuration file typically in /etc, which means when we’re asked questions about such configuration we need to ensure that the cached data we have still reflects the actual data in the file.
Even if not available today, this is probably quite easy to support with the infrastructure we already have inside snapd, since it’s very similar to the filtering we do on the configuration when we get new settings coming in. That is, the configure hook today is able to take data that is being changed, reject it, but also adapt as it wishes, before it gets saved. We need a hook doing something similar on the way out: get the data, allow the hook to update the current knowledge, and get the new data saved and available for the query.
As for the use case at hand, my suggestion is for us to go ahead and implement the support using what Jon suggests above, or some variation of it, since the actual feature you need is coming soon. For example, in addition to the nodes being triggered to update the configuration, if you want to be extra paranoid, the node itself could also trigger an update every time k8s get/set is run, just in case. Once snapd gets updated with the feature, both of these workarounds can be dropped in favor of the actual hook.
In any case, I’d strongly recommend not reinventing the wheel. There’s a lot of logic behind get/set, and you’ll continue to get support from the snapd team to get this working well for you, vs. having a large feature that you need to develop/fix/support yourself.
oops, of course… I meant 