Snapd 2.37.1 refresh broke layouts snap remove/install


#1

I’m not totally sure about the sequence of operations here, but my understanding is that basically the following happened to me:

  1. A snap was locally installed that used layouts a week or two ago with snapd 2.36.3 (and was working fine from the standpoint of using layouts) The layout spec looks like this
passthrough:
  layout:
    /etc/snap-name:
      bind: $SNAP_DATA/etc/snap-name
  1. Machine is rebooted today
  2. snapd refreshes to 2.37.1
  3. The snap in question is explicitly removed with snap remove snap-name
  4. A new local version of the snap is installed locally (no change in layout spec, just in snap contents)
  5. The mount exists creating /etc/snap-name, but I couldn’t create files, directories or anything in that directory:
$ sudo snap run --shell snap-name.svc
# ls -lah /etc/snap-name
total 0
# mkdir /etc/snap-name
mkdir: cannot create directory '/etc/snap-name': File exists
# mkdir /etc/snap-name/tmp
mkdir: cannot create directory '/etc/snap-name/tmp': No such file or directory
# touch /etc/snap-name/hello
touch: cannot touch '/etc/snap-name/hello': No such file or directory

I see the following in /proc/self/mountinfo for that snap:

# cat /proc/self/mountinfo | grep etc/snap-name
127 186 252:0 /var/snap/snap-name/x1/etc/snap-name//deleted /etc/snap-name rw,relatime master:1 - ext4 /dev/mapper/xenial--classic--vg-root rw,errors=remount-ro,data=ordered

Specifically that //deleted looks very suspicious to me…

  1. After rebooting, now layouts works for this snap as expected.

This was all done on an Ubuntu Server 16.04 virtual machine and I didn’t see anything suspicious in journalctl --no-pager -e.

This was the revision history of the core snap:

$ snap list core --all
Name  Version    Rev   Tracking  Publisher   Notes
core  16-2.36.2  6034  stable    canonical✓  core,disabled
core  16-2.36.3  6130  stable    canonical✓  core,disabled
core  16-2.37.1  6350  stable    canonical✓  core

Ping @zyga


#2

EDIT:

Some questions:

  • Was this a try-mode snap?
  • Was this done inside a container?

I have not attempted reproducing this yet but this is important difference.


#3

No this was not a try-mode snap. It was installed with --devmode if that’s relevant

No, the machine was a full virtual machine running with VirtualBox


#4

So I ran into this again with the same snap but without a snapd refresh so I think this is just a bug with layouts and not anything specific to the 2.37.1 release.


Snap layouts
#5

This is potentially related to https://bugs.launchpad.net/snapd/+bug/1815722


#6

Since the associated PR was merged, I will test this when it lands on the edge channel of snapd and see if I can reproduce it still.