Hi,
I have a question: Let’s say I build a custom image and make changes to configuration files, e.g. in my case /etc/sshd_config. What happens when I update the system or parts of it (e.g. the core snap)? Will my configuration file be overwritten or is it handled like in other distributions, e.g. by keeping the customized file intact and placing the new file as sshd_config.new in the same folder?
Kind regards,
Michael
the simple answer is that you cant 
the rootfs of a core image is the core snap, this is a readonly, compressed and gpg signed squashfs and contains only the minimal bits to make your system boot (everything else has to come from a snap package). if you change the core snap manually, your image is not vaid, and yes, any upgrade, security fix, auto-rollback etc that the image performs in operation will expect that your core snap is the validly signed snap package the store provides (while it is technically possible to re-build core manually, this is typically only used to verify patches before upstreaming them into the official tree).
what you would do in a case where you want a custom ssh setup is to have your custom gadget completely disable the core shipped ssh, then create an ssh snap and have that ship your custom config (and run the daemon from inside the snap).
Hi ogra,
thank you!
I have some follow-up questions.
1.) If I create a special gadget snap and I want a user that is connected via SSH to be able to read / write files in the file system outside the snap, then I would have to connect all the required interfaces, right? In this case this would mean that we would have to connect lots of interfaces (because we might want to login to SSH and then using sudo to gain root privileges)
2.) If a have a snap which has the permission to read/write to /etc (using the system-files interface), then I could use this snap write a custom sshd_config file and to monitor changes (after an update of the core snap) and restore the custom configuration, right? Would this be an approach that you could recommend? Sounds a bit hacky to me 
3.) Just out of interest: why is /etc/ssh/ writable when the files are overwritten, anyway?
Kind regards,
Michael
indeed that wouldnt work with such a snap (sudo within snaps would require a new interface that allows access to the hosts sudo setup)
you would need a brand store for this, i doubt a snap accessing sshd_config at will would be allowed into the global store at all
because ubuntu corer needs to create fresh host keys on first boot, else every ubuntu core device would have the same fingeprint
i actually dont think either of teh above is a good solution, perhaps cloud-init (despite also a bit hackish given it isnt designed to be used in standalone appliance installs) might be the answer here …
what exactly do you want to change in the config btw ?
Hi ogra,
Then this solution won’t work for us, unfortunately.
We have a brand store, so that wouldn’t be the problem. But as you mentioned it would still be a hacky solution anyway.
Yeah, that sounds plausible. 
On our system the main interface is web based, but we want to be able to configure that a specific (configurable) user (or group) is able to connect via SSH to the system. To enable this, my idea was to create a group (e.g. “sshusers”) and change the SSH configuration to only allow members of this group to connect via SSH. I already tested in on a live system and it worked - but a colleague then mentioned that there will be a problem when updating the snap (which was the reason to open the thread here - in the end you confirmed what he said).
sshd only accepts configuration files from /etc/ssh, right?
One question which a colleague brought up: He just tested the scenario (custom image, snap update) and the file was not overwritten. Nevertheless, he suspects that this behavior is related to the delta update mechanism and that the sshd_config would be overwritten at the latest when the file content changes from one snap revision to another. Is his assumption correct?
Kind regards, Michael
I don’t think this is entirely accurate.
On UC systems, /etc/ssh is really bind mounted from /system-data/etc/ssh (which you can find while running from an SSH shell at /writable/system-data/etc/ssh). You can confirm this by inspecting /proc/self/mountinfo:
$ cat /proc/self/mountinfo | grep etc/ssh
34 25 179:2 /system-data/etc/ssh /etc/ssh rw,relatime shared:13 - ext4 /dev/mmcblk0p2 rw,data=ordered
Since specifically /etc/ssh is bind mounted from the system-data folder, AIUI you can modify that etc/ssh/sshd_config file when you build the image and that file should persist as long as nothing that runs outside of confinement modifies it. Indeed I can’t find any interfaces in snapd that would permit modification of this file, so you wouldn’t be able to modify this file from inside a snap. As a general rule however, @ogra is correct in that the root file system comes from the core snap and as such can’t be modified, but some directories such as /etc specifically are bind mounted from the image/host filesystem on top of whatever is in the core snap. See https://github.com/snapcore/snapd/blob/master/cmd/snap-confine/mount-support.c#L566-L577 for more details.
@ijohnsonis indeed right, this thread made me check the writable-paths again:
$ grep /etc/ssh /etc/system-image/writable-paths
/etc/ssh auto persistent transition none
i thought i remembered it being “synced” … sorry for the confusion … so sshd_config will not be overwritten by an upgrade/rollback
… and since you have a brand store using a snap that uses the system-files interface and alters (or replaces) the sshd_config file should be possible from that … assuming you have an appliance configuration snap installed anyway to set your defaults, you can simply add some scriptery there to adjust sshd_config to your liking.
Hi,
thank you for both the update! This sounds very promising and makes the realization of our specific use case a lot easier!
I assume that the behavior wont’t change in future versions, right? Because then we will include the sshd_config directly in our custom image. Nevertheless, the approach by using the system-files interfaces to access files in /etc sounds interesting for some other use cases that we have.
Kind regards,
Michael
i’d really go with a configuration snap for this from the start, that way you can actually update the changes in the image at runtime if needed … but it is indeed up to you