Cmake breaks after upgrade due to references to `/snap/cmake/340/...`

At one point my cmake binary was located at /snap/cmake/340/bin/cmake , causing my Makefile to be littered with hard-coded references to that path. Lo and behold, some update along the way must have changed the path, and now it complains that cmake is not found. Why couldn’t you use /usr/bin/cmake with symlinks?

snap version
snap 2.45.2
snapd 2.45.2
series 16
ubuntu 18.04
kernel 5.3.0-53-generic

Edit: Thank you @galgalesh for framing the question in a more constructive way. I appreciate the work all of you do here!

1 Like

I think what should happen here is that cmake uses /snap/cmake/current/bin/cmake. This will always point to the latest version. I’m not sure why but the environment variables like $SNAP and $SNAP_USER_DATA use the real path instead of the symlinks. This breaks a number of things after updates. Many of my snaps use workarounds to fix this issue.

I think snapd can make stuff a lot easier for maintainers if it points $SNAP to the symlink instead of the real path.

1 Like

Pinging @crascit since he maintains this snap.

1 Like

It would be nice is current was not a symlink but an actual mount instead. Is this something that can be looked into @mvo?

1 Like

Ideally it should be using /snap/bin/cmake rather than any direct paths under $SNAP.

If cmake wasn’t classic confined, this kind of thing would represent a sandbox escape for the cases when the paths didn’t break over an upgrade.

2 Likes

What are the advantages of current being a mount instead of a symlink? As @jamesh said, apps really shouldn’t be looking at things there “from the outside”, as this represents either confinement escape for the snap, or at best a broken app since the app is now running with the host’s environment rather than the snap’s base.

Most applications “resolve” current, even snapd when setting up the environment:

sergiusens@umbar:~$ snap run --shell beets.beet
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

sergiusens@umbar:/home/sergiusens$ echo $HOME
/home/sergiusens/snap/beets/6
sergiusens@umbar:/home/sergiusens$ echo $SNAP
/snap/beets/6

That is my personal pain point as I have to navigate code that writes configuration so that the configuration writing logic is snap aware.

For the case of cmake, it causes things like this (as seen from the cmake configuration logs):

  ignore line: [/usr/bin/c++   -v -o CMakeFiles/cmTC_170b6.dir/CMakeCXXCompilerABI.cpp.o -c /snap/cmake/513/share/cmake-3.18/Modules/CMakeCXXCompilerABI.cpp]

And I am not sure why, cmake eventually resolves to its real self:

  ignore line: [/snap/cmake/513/bin/cmake -E cmake_link_script CMakeFiles/cmTC_170b6.dir/link.txt --verbose=1]

If the concern is that updates would affect AppArmor profiles and current, I would also like for the “do not update underneath me while running” to also get some attention :slight_smile:

I think the error you are encountering here is more about the definition of various environment variables like $SNAP and $SNAP_DATA including the revision number, if they included just current, then things would work better for you I think.

Last I recall, the main blocker before we can safely make that change is to get refresh app awareness implemented and non-experimental, as currently we use revision numbers so that in the case where an app is refreshed behind it, it is not totally broken (just erm a bit broken). With refresh app awareness and changing these env vars to use current I think you would see a lot less of these types of errors.

It’s not just those variables, because if you canonicalise a path, which some apps do (such as gcloud when setting the path to the kubernetes authentication helper - see here) then they’ll get the path with the revision number embedded:

$ snap run --shell gimp
$ readlink -f $SNAP/usr/bin/gimp
/snap/gimp/281/usr/bin/gimp-2.10
$ readlink -f /snap/gimp/current/usr/bin/gimp
/snap/gimp/281/usr/bin/gimp-2.10
2 Likes

Hmm, we would need to evaluate the implications of using a mount for current instead of a symlink, one potential problem is when we perform the change between one revision and the next, we do this change as snapd, which runs outside of the snap’s mount namespace and we would need to be careful to do it in such a way that it is inherited properly by the snap when it runs, as snap-update-ns is run later, and we probably don’t want snap-update-ns to update which revision is current as that is too late for other tasks we do.

It would be helpful if someone started either a LP bug or a new post in #snapd detailing these issues with canonicalization with examples of programs that do this canonicalising and then run into problems (gcloud being a good example probably).

In CMake’s case, one of the things it has to do is store a path to itself in the build rules. If certain dependencies change, the build automatically re-runs cmake for you. That’s likely what the original poster has experienced. Looking at one example build, I see that the full versioned path is getting stored in the build rules. I am invoking cmake as /snap/bin/cmake so something along the way is causing CMake itself to either see the versioned path when it finds itself, or it is resolving the path to itself. There has been recent work in the past few years to try to avoid resolving paths for other things like the path to the source or build directory, but I don’t know if that extends to the logic used for CMake working out the path to itself. I’ll have to do some investigating and get back to you.

Confirmed, it is CMake that is resolving symlinks. I’ve created an issue for it in the CMake issue tracker here:

https://gitlab.kitware.com/cmake/cmake/-/issues/21059

1 Like

The meta/snap.yaml file for your snap lists:

apps:
  cmake:
    command: bin/cmake
    completer: share/bash-completion/completions/cmake
    command-chain:
    - snap/command-chain/snapcraft-runner

So after entering the sandbox, /snap/bin/cmake will execute the following:

$SNAP/snap/command-chain/snapcraft-runner $SNAP/bin/cmake

The snapcraft-runner script chains to the next with exec "$@". So when the real cmake executable is executed, it would see the full /snap/cmake/NNN/bin/cmake path name. It looks like you’d get the same argv[0] value without the command chain too.

You’d probably need some snap-specific code path to pick the wrapper. Getting this right in the presence of command aliases is non trivial, but probably not a big deal if “cmake” is the only command that gets encoded into makefiles.

My initial reaction is that the onus should be on snapd to not change the way the app is invoked. It is preventing the real underlying app from being able to detect the original command path used to invoke it. That is fundamentally changing the app behavior in a way that the app itself cannot easily handle. Worse, snapd doesn’t even seem to provide an environment variable with that information for the app to check for as a special case.

I am hesitant to patch the CMake source code with what I suspect would be fairly fragile logic to try to infer what the original invoking command might have been. It shouldn’t be necessary in the first place. I expect that there are plenty of other snaps which have had to work around this, so can we not have a cleaner way to deal with this situation?

1 Like