Layouts: re-mapping snap directories

So, some good news and some updates on where we are and what’s ahead:

  • The good news, layouts are now in master / edge.
  • The bad news, a sanity check spread test doesn’t work on core devices yet, I’m investigating this (https://github.com/snapcore/snapd/pull/4644)
  • The security news is that layout mechanics needs to go through a hardening phase where snap-update-ns will come with an apparmor profile tailored to a given snap. This is my 2nd priority item after the spread test above
  • I have one more branch that ensures layouts don’t construct partially (https://github.com/snapcore/snapd/pull/4658).

So the big picture is that things are mostly there for layouts. If you are keen to try them out you can grab master, look at the spread test for inspiration and start experimenting.

I’m still due to write a documentation chapter and that will be my focus after the code is ready. I will be taking a small detour to help land the per-user mount namespaces to help the desktop team with XDG portal support so I cannot say exactly how timing looks like.

3 Likes

Two PRs that fixed a bug affecting symlinks and read-only directories and layouts on core were merged both into master and into the 2.32 release branch. With those fixes layouts should now operate on core and classic without any major issues.

I will look into updating the documentation and I’m looking forward to 2.32 :slight_smile:

2 Likes

Last week I identified two bugs that affect layouts:

  • layouts that use symlinks cannot cope with the symlink already being present and correct
  • layouts can leak some empty files, empty directories or symlinks into the non-snap space

Both of the issues are in progress, the first is under review at https://github.com/snapcore/snapd/pull/4851 and the second one is under development.

I was also made aware of an issue when refreshing try-mode snaps that are using layouts and I am investigating the issue. It looks like a bug in the update algorithm. I will post more updates as I get to the bottom of the issue.

Given that layouts are still a bit experimental we have decided to put them under feature flag.

To use layouts in 2.32 you will need to enable it with snap set core experimental.layouts=true.

1 Like

One small issue I hit during my experiment with (still experimental) layouts (discussed this on IRC, mentioning here so we don’t forget):

I was trying to define a layout such as this:

/usr/lib/tcltk/x86_64-linux-gnu/tk8.5:
  bind: $SNAP/usr/lib/tcltk/x86_64-linux-gnu/tk8.5

but got the following error when running my app:
cannot update snap namespace: cannot create writable mimic over "/usr/lib": permission denied;

and the following denial:
[68891.838823] audit: type=1400 audit(1524071502.757:492): apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13 profile="snap-update-ns.scid" name="/tmp/.snap/usr/lib/" pid=15755 comm="3" srcname="/usr/lib/" flags="rw, rbind"

It seems that the directories prior to “tk8.5” must exist for layout to work - I got it working by bind-mounting everything at the /usr/lib/tcltk level.

Thanks for reporting. That sounds like something we need to fix indeed.

@zyga-snapd Do you know what went wrong there?

Yes, when we grant permissions for snap-update-ns to construct a writable mimic we do so for the directory that was specified in the layout. The problem is that here we, at runtime, notice that /usr/lib/tcltk/ doesn’t exist and so decide to make a mimic at /usr/lib.

One way of fixing it would be to use the container work from Chipaca to inspect the base snap and know what we are dealing with. I need to think about other approaches.

@zyga-snapd There’s something a bit suspect in this constraint if all we are doing is allowing whatever the snap asks for. In other words, so we protect the environment from mounting into /usr/lib, unless the snap actually asked for it? Why?

We don’t grant arbitrary mounts so we grant what the snap asks for. This is not specific to /usr/lib in any way.

Right, the point is that if the snap asks for /foo/bar/baz, we allow it. If it asks for /foo/bar, we allow it… so this is not strong security constraint in the sense we are actually allowing both, right? And now, apparently allowing /foo/bar/baz but not /foo/bar means things are breaking down. So why not simply allowing the parents as well?

1 Like

We disallow certain things so I need to think how to make us allow what is safe without breaking the earlier constraints.

I sent a PR summarising the problem and showing what the permissions we hand out are. I made a suggestion that follows what Gustavo suggested above and I will be making subsequent modifications but I need to discuss this with @jdstand briefly.

EDIT: PR is actually https://github.com/snapcore/snapd/pull/5074

FYI, I commented in the PR and came to the same conclusion as @niemeyer.

I’ve implemented the proposed solution. I will push it shortly (just waiting for local run of spread with the extended layout test).

I have this fixed now and I will be making one or two PRs that introduce the fix. I also found one more interesting bug related to layouts in particular. I will summarise the details in the PRs and link them here.

EDIT: The 1st “PR” for chopTree is https://github.com/snapcore/snapd/pull/5081 (more things on the way for this effort though)
EDIT: The 2nd “PR” for the poke-hole in the source path for layouts: https://github.com/snapcore/snapd/pull/5090
EDIT: The 3rd “PR” for the mimic tmpfs mode and ownership: https://github.com/snapcore/snapd/pull/5107

I have omitted small PRs that just make the place for those three available. I will update the 1st link to refer to the actual fix (this is just a function for review so far) once that is available for review.

Hey, any word on when this will become officially supported (as opposed to experimental)?

My gut feeling is that after one full cycle without known issues. Currently there’s still one^Htwo issues (aka “trespassing” bug and “chopTree” bug) that I’m working on (it’s a deeper problem than initially thought).

2.35 maybe?

I’m very much interested in this, and hope it will be available soon! Good work to all!

1 Like

2.34 should have one (or with some luck) both of the remaining fixes merged. In any case the issues are very isolated and non earth-shattering so you can try to use layouts to make your snaps easier today.

2 Likes

Awesome, thank you! :clap:t2:

@zyga-snapd Did you mean 2.34 or 2.33.1? Because right after your answer, I have seen a 2.33.1 release, but there is still no 2.34! Sorry if I misunderstood you…