ROS2 python package in snaps

snapcraft
1

Hi,
How to handle python libraries such as pyserial in ros2 python packages, when turning them into snaps

2

Hey there, @MJalloh, welcome to the community! This is a perfect place to move our conversation (for interested parties, it originated here). I’m going to duplicate a bit of the discussion here just so that link isn’t required reading to understand this.

In general, the way to get external python libraries into your ROS snap is the same way you’d get them as a dependency when blooming the ROS package: add them to the package.xml as a dependency. This requires the library to be present in the rosdep ruels. If it’s not already there, it’s a pretty easy pull request to make (something that happens all the time).

In the specific case of pyserial, it’s already there, called python-serial. That’s the key upon which you’d depend in the package.xml. On Ubuntu, for ROS 2 distros (which are python 3), it would resolve to the python3-serial Debian package, which does indeed appear to be pyserial:

$ apt-cache show python3-serial
<snip>
Source: pyserial
3

Trying its out and see

4

it worked, but the new issue am is an import error

File “/snap/ros2-rover/x1/usr/lib/python3/dist-packages/numpy/core/init.py”, line 16, in
[controller-3] from . import multiarray
[controller-3] ImportError: libblas.so.3: cannot open shared object file: No such file or directory

5

This is at runtime, right? Not at build time? In that case, try adding the environment property to the app that extends the LD_LIBRARY_PATH a little, something like this:

# ...
apps:
  my-app:
    command: my-command
    environment:
      LD_LIBRARY_PATH: "$LD_LIBRARY_PATH:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/blas:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/atlas"
# ...

Note that I’m not sure if you’re using libblas3, or libatlas3-base, or both, so I added both paths there. If you notice that only one of them exists you can remove the other. The point is, though, that snapcraft doesn’t always find libraries within subdirectories of the common paths and add them to LD_LIBRARY_PATH on your behalf (e.g. the blas subdirectory of the /usr/lib/<triplet> common path).

6

@sergiusens, @cjp256 I seem to remember snapcraft doing a decent job of this because of the elf crawling, but I’m a bit fuzzy on it nowadays. Is this happening because it’s a dependency from python?

7

Needed to add lapack too ended up with but still can’t add python3-serial in the package

environment:
      LD_LIBRARY_PATH: "$LD_LIBRARY_PATH:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/blas:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/atlas:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/lapack"~~~
8 
xxxxxxx@xxxxx:~/ROS/ros2_workspace$ sudo rosdep install -i --from-path src --rosdistro $ROS_DISTRO -y
[sudo] password for xxxxxxx: 
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
rover: Cannot locate rosdep definition for [python3-serial]
9

package yaml

<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
  <name>rover</name>
  <version>0.0.0</version>
  <description>TODO: Package description</description>
  <maintainer email="">mjalloh</maintainer>
  <license>TODO: License declaration</license>

  
  <exec_depend>rclpy</exec_depend>
  <exec_depend>sensor_msgs</exec_depend>
  <exec_depend>python3-serial</exec_depend>
  <depend>joy</depend>
  

  <test_depend>ament_copyright</test_depend>
  <test_depend>ament_flake8</test_depend>
  <test_depend>ament_pep257</test_depend>
  <test_depend>python3-pytest</test_depend>

  <export>
    <build_type>ament_python</build_type>
  </export>
</package>
10

The rosdep key is python-serial, not python3-serial. rosdep will look that up, notice you’re using a ROS distro that uses Python 3, and resolve it to the Debian package python3-serial. Your package.xml line still needs to use the key, though:

<exec_depend>python-serial</exec_depend>
11

Elf crawling is for classic confinement only, we can discuss enabling rpath mangling for non classic.

12

I’m not talking about the rpath mangling (which does indeed happen only for classic), I’m talking about tracking part dependencies by interrogating its ELF files, which happens for app snaps. That information should turn into a path contained within LD_LIBRARY_PATH here. It’s not working in this case, but I’m thinking it’s because this lib is loaded from python instead of a C library.

13

Create a new package

ros2 pkg create --build-type ament_python rover2 --dependencies rclpy python-serial

then installed with

sudo rosdep install -i --from-path src --rosdistro $ROS_DISTRO -y

then build the package
then loaded with

. install/setup.bash

but when I run it. still gets an error

 from rover2.py import Py, findArduinoPort
  File "/home/mchael/ROS/ros2_workspace/install/rover2/lib/python3.6/site-packages/rover2/py.py", line 2, in <module>
    import serial.tools.list_ports
ModuleNotFoundError: No module named 'serial'
14

it only works if i install python3-serial with apt-get

15

When you run that, what does it install? Is it actually installing the python2 version?

16

Huh, indeed:

rosdep resolve python-serial --os ubuntu:bionic --rosdistro eloquent
#apt
python-serial

This seems like a bug. I’m not sure if it’s in rosdistro or rosdep, though. Might be worth a forum post on the ROS Discourse. I thought maybe if I set ROS_PYTHON_VERSION=3 it would change things, but it doesn’t seem to do anything.

17

I think I’ve confirmed that this is a bug, probably in rosdep. We might be able to work around it by writing another rosdistro rule, but I’ve asked for clarification first.

Update: Yep, seems like a new rule is required. I’ve added it here. As soon as it lands you should be able to build your snap by using python3-serial as the dependency name in your package.xml.

18

Thanks man. Will be waiting.

19

Sorry for the late reply. Yeah python-serial for python2 is been installed. Just tested it

20

How long before I can test it