Support for cgroup v2

abitrolly · January 24, 2021, 11:09pm

snapd doesn’t work properly on a Linux with newest and greatest cgroups v2. At least it always shows this warning.

WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement

What are the plans to get full support for v2 and what are the major obstacles?

mborzecki · January 25, 2021, 7:44am

The major obstacle right now is supporting the device controller. In cgroup v1, it was controlled by writing textual information about a device to a file under the cgroup controller. With v2, the files are gone, and one needs to attach an eBPF program that implements the functionality. Hopefully it’s as simple as concatenating a static prologue/epilogue with a bunch of direct jumps in between.

As for some easier things still in the TODO list, there’s missing support for the freezer, which is no longer a separate controller, but there’s a single cgroup.freeze file that you write to. The support needs to be added to snap-confine and snap-update-ns (via sandbox/cgroup package).

This bug contains more information about timelines: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1850667

The patches are always welcome, feel free to open PRs or ping us on IRC if you wish to help.

abitrolly · January 25, 2021, 12:37pm

I think it is better to continue here, to answer questions for more people than it would be on IRC.

What this device controller is doing? What does it mean to support it in snapd? eBPF looks like a programming language for filtering network packets. I don’t see how all of this plays together.

For the freezer there is already a freezer.state file that is being written.

github.com

snapcore/snapd/blob/aef99b4557d02cfc1c52b5628a6a75071a5a6af2/sandbox/cgroup/freezer.go#L65


// ThawSnapProcesses resumes execution of all processes belonging to a given snap.
//
// This operation can be mocked with MockFreezing
var ThawSnapProcesses = thawSnapProcessesImpl
// freezeSnapProcessesImpl freezes all the processes originating from the given snap.
// Processes are frozen regardless of which particular snap application they
// originate from.
func freezeSnapProcessesImpl(snapName string) error {
	fname := filepath.Join(FreezerCgroupDir, fmt.Sprintf("snap.%s", snapName), "freezer.state")
	if err := ioutil.WriteFile(fname, []byte("FROZEN"), 0644); err != nil && os.IsNotExist(err) {
		// When there's no freezer cgroup we don't have to freeze anything.
		// This can happen when no process belonging to a given snap has been
		// started yet.
		return nil
	} else if err != nil {
		return fmt.Errorf("cannot freeze processes of snap %q, %v", snapName, err)
	}
	for i := 0; i < 30; i++ {
		data, err := ioutil.ReadFile(fname)
		if err != nil {

Is there a different file or a different content that should be written here?

mborzecki · January 25, 2021, 12:40pm

If you’re interested, I suggest looking at the kernel documentation first: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html as for BPF, try the manpage bpf(2) as a starting point. Also, Fedora ships with cgroup v2 by default, so spinning up a VM (rawhide or 33) to play around is useful too.

James-Carroll · January 25, 2021, 12:45pm

eBPF isn’t really “extended BPF” anymore, it’s functionality has gone well beyond just packet filtering rules. The way I’ve read it is it’s better to just treat eBPF as its own buzzword rather than a literal extension to BPF. I think here is a good overview

abitrolly · January 25, 2021, 1:05pm

Basically, the whole reason I started this thread is because I am on Fedora. ) It was not problematic until I got into problem with packaged Electon app (matrix.org client) where it is not clear if the problem is in packaged software itself, its dependencies or in the incomplete cgroup v2 support in snapd.

mborzecki · January 25, 2021, 1:06pm

What is the problem with matrix client then and why do you think it’s related to cgroups?

abitrolly · January 25, 2021, 1:11pm

That would be offtopic here, so here the link to matrix related thread. Electron-builder produces freezing snaps

I thought that cgroup may be involved, because the log of errors on stdout starts with this big warning that cgroup v2 is not full supported yet. It is shown every time a snap is started, but this time the app could not even show a window.

mborzecki · January 25, 2021, 2:00pm

Unless it’s connecting and disconnecting interfaces that update the mount ns, I would rather look somewhere else for the cause of the feezes. Maybe there’s some tracing facilities built into electron?