Encountering problems when packaging RViz

[defunctzombie]

I am packaging rviz as a snap and running into some issues.

My snapfile.yaml:

name: rviz 
version: 1.11.14
summary: rotobot operating system framework visualizer 
description: | 
 3D visualizer for use with ROS framework 

grade: stable 
confinement: devmode 

parts: 
 rviz-workspace: 
   after: [desktop-qt4] 
   plugin: catkin 
   rosdistro: indigo 
   catkin-packages: 
     - rviz 

apps: 
 rviz: 
   command: desktop-launch rviz 
   plugs: [network, network-bind, home, x11, opengl]

You can use the above file by following these steps:

mkdir workspace
cd workspace
mkdir snap
// copy the above snapcraft.yaml into the snap dir
mkdir src
cd src
git clone https://github.com/ros-visualization/rviz.git 
cd rviz
git checkout -t origin/indigo-devel
cd ../..
snapcraft cleanbuild

Basically, the above is the layout that will allow you to use the catkin snap plugin to build rviz.

For me the build runs to completion (rviz is built and its install steps are run). However it then fails at what I presume is the snap packaging phase with the following:

Cleaning up newly installed Catkin packages...
Parts 'desktop-qt4' and 'rviz-workspace' have the following file paths in common which have different contents:
   etc/fonts/conf.avail/10-no-sub-pixel.conf 
   etc/fonts/conf.avail/10-scale-bitmap-fonts.conf  
   etc/fonts/conf.avail/30-metric-aliases.conf      
   etc/fonts/conf.avail/45-latin.conf              
   etc/fonts/conf.avail/60-latin.conf     
   etc/fonts/fonts.conf                     
   etc/init.d/x11-common
   lib/x86_64-linux-gnu/libexpat.so.1.6.0
   usr/bin/fc-cache    

... and so on

Snapcraft offers some capabilities to solve this by use of the following keywords:
   - `filesets`                           
   - `stage`        
   - `snap`    
   - `organize`

I looked into the above keywords however the documentation didn’t help me too much in understanding how to properly apply them to my problem.

Any tips or guidance in what I should try next would be much appreciated.

RViz is a Qt4 application that needs opengl.

[defunctzombie]

The above snapfile.yaml was created by assembling ideas from these posts:

• http://wiki.ros.org/ROS/Tutorials/Packaging%20your%20ROS%20project%20as%20a%20snap
• https://insights.ubuntu.com/2016/07/06/ubuntu-app-developer-blog-announcing-new-snap-desktop-launchers/

[kyrofa]

Hey there @defunctzombie, thanks for posting here!

The issue actually comes down to the rosdistro you’re using (Indigo) and the Ubuntu version on which you’re building (Xenial). Since Indigo isn’t available in the ROS archives for Xenial, the Catkin plugin actually uses the Trusty repos. Unfortunately for you, that actually means you’re mixing repo sources in this particular YAML: desktop-qt4 will actually be satisfied by the Xenial repos, and rviz by the Trusty repos, which means if they happen to use any package that differs between the two repos, you’ll have conflicts (which is to be expected, and is the case here: fontconfig is different, looks like Qt4 is different, and so on). That means that each part (desktop-qt4 and rviz) are trying to stage the same files, but each set of files comes from different versions of the package, and Snapcraft doesn’t know which ones you want to stage!

Snapcraft offers the stage and prime keywords that can be used to filter out select files from specific parts-- one of its uses is for exactly this case. So you could filter out conflicting files from, say, the rviz part with something like this (note the minus signs indicating that these files should not be staged):

# ...
 rviz-workspace:
   after: [desktop-qt4]
   plugin: catkin
   rosdistro: indigo
   catkin-packages:
     - rviz
   stage:
     - -etc/fonts/conf.avail/10-no-sub-pixel.conf 
     - -etc/fonts/conf.avail/10-scale-bitmap-fonts.conf
# ...

However, that assumes that Trusty’s rviz will run just fine against the libs in Xenial, which may very well not be the case. If you need to use Indigo (as opposed to Kinetic), for a snap like this you honestly probably need to build on Trusty. This is harder than it sounds right now, I’m afraid. We’re super close to having the snap work there which will make things painless, but you can prototype by following this guide.

I’ll post back here as soon as the snap suitably runs on Trusty. The PRs have been proposed, they just need to land.

[defunctzombie]

The issue actually comes down to the rosdistro you’re using (Indigo) and the Ubuntu version on which you’re building (Xenial).

I am trying to use Indigo however I am building on Artful (as my host). I am using cleanbuild. I do see that Xenial apt sources are fetched during the build process and was concerned about this but did not find a way to force it to use Trusty (not sure where it gets Xenial from because I did not configure that).

I will try using the Kinetic distro as I believe none of the messages changed.

One thing that isn’t clear to me is where I need to build my snap. If I want to use the snap on Artful, I need to build on Artful (but it seems that xenial deps are downloaded in cleanbuild). If I want to use it on Trusty, I would need to build it on Trusty?

[kyrofa]

Series 16 snaps (the only type of snaps there are right now) run against a Xenial-based core snap. As a result, things tend to work best when that’s also your build host. Using snapcraft cleanbuild will indeed use Xenial containers. The only way to get Trusty repos right now is to actually build on Trusty (and not use cleanbuild, because that would use Xenial again). Building on releases earlier than Xenial (not using cleanbuild) tends to work okay because the libc in the core snap will still satisfy the requirements, but building later than Xenial (again, not using cleanbuild) can run into issues.

Note that this same core snap (xenial-based) is used when running snaps on Trusty as well.

[defunctzombie]

I got farther by changing the distro to Kinetic, the snap builds. I now have new errors when I try to run it,

[ INFO] [1509399213.079343954]: rviz version 1.11.14 
[ INFO] [1509399213.079375946]: compiled against Qt version 4.8.7 
[ INFO] [1509399213.079385518]: compiled against OGRE version 1.9.0 (Ghadamon) 
[ WARN] [1509399213.304159180]: OGRE EXCEPTION(7:InternalErrorException): Could not load dynamic library /root/build_rviz/parts/rviz-workspace/install/usr/lib/
x86_64-linux-gnu/OGRE-1.9.0/RenderSystem_GL.  System Error: /root/build_rviz/parts/rviz-workspace/install/usr/lib/x86_64-linux-gnu/OGRE-1.9.0/RenderSystem_GL.s
o.1.9.0: cannot open shared object file: Permission denied in DynLib::load at /build/ogre-1.9-mqY1wq/ogre-1.9-1.9.0+dfsg1/OgreMain/src/OgreDynLib.cpp (line 109
)                                                                                                                                                               
terminate called after throwing an instance of 'Ogre::InternalErrorException' 
 what():  OGRE EXCEPTION(7:InternalErrorException): Could not load dynamic library /root/build_rviz/parts/rviz-workspace/install/usr/lib/x86_64-linux-gnu/OGRE
-1.9.0/RenderSystem_GL.  System Error: /root/build_rviz/parts/rviz-workspace/install/usr/lib/x86_64-linux-gnu/OGRE-1.9.0/RenderSystem_GL.so.1.9.0: cannot open 
shared object file: Permission denied in DynLib::load at /build/ogre-1.9-mqY1wq/ogre-1.9-1.9.0+dfsg1/OgreMain/src/OgreDynLib.cpp (line 109) 
Aborted (core dumped)

When I inspect the snap using “unsquashfs -l” I see the following file is present:

squashfs-root/usr/lib/x86_64-linux-gnu/OGRE-1.9.0/RenderSystem_GL.so.1.9.0

However, runtime cannot find it (presumably some LD_LIBRARY_PATH or other dynamic loading issue).

I will try adding to the LD_LIBRARY_PATH in my snapcraft.yaml file and see if that resolves the issue however I am not confident as it looks like it is trying to load a hard-coded path.

[kyrofa]

Try this just to make sure it works, then we can make it more architecture-friendly later:

apps: 
  rviz: 
    command: desktop-launch rviz 
    plugs: [network, network-bind, home, x11, opengl]
    environment:
      LD_LIBRARY_PATH: "$LD_LIBRARY_PATH:$SNAP/usr/lib/x86_64-linux-gnu/OGRE-1.9.0"

[defunctzombie]

Did not work.

Same error when I run the snap.

I also checked that the LD_LIBRARY_PATH was being set (to rule that out) by using snap run --shell rvz and doing env | grep LD

LD_LIBRARY_PATH=:/snap/rviz/x1/usr/lib/x86_64-linux-gnu/OGRE-1.9.0

My guess is that something during the build hard-codes a path to where this library should be found (or there is some other environment variable that must be set).