2
$\begingroup$

(ROS2 HUMBLE, JETSON XAVIER NX, DOCKER CONTAINER)

I have a RealSense camera running in a Docker container on my Jetson Xavier NX. The camera publishes RGB images, depth images, point clouds, and IMU data.

When I run the command ros2 topic hz in my docker container, I can see that the data is being published at approximately the exact rate specified in my parameters file.

However, when I try to visualize the images or point cloud on my computer using RViz, the frame rate drops drastically, almost to zero.

While I expected some reduction in rate due to the size of the data and the fact that it is being transmitted over Wi-Fi, I didn't anticipate such a severe drop.

How do people typically resolve this issue? is the compressed images the only way to go?

Thank you.

$\endgroup$

1 Answer 1

1
$\begingroup$

Tl;dr: use webrtc for video and reduce/sub-sample your pointclouds on the robot before transmission, or stream a video of the visualized pointcloud.

Both images and point-clouds are huge when uncompressed. Fortunately for images, there are very good video-compression algorithms which are a must-have for streaming images/video remotely. We wrote a comparison on the various methods here.

What it boils down to is, that for images/video you really want to use webrtc which utilizes h264 or similar video compression technology. This easily reduces the size you need to transmit 100x or more.

Pointclouds, unfortunately, are much harder payload to work with. While there are pointcloud compression techniques like Draco they are not nearly as efficient as the video compression techniques above. Part of the problem is that with pointclouds you typically want a lossless compression, which you are typically willing to give up for video when it is meant for human/visual consumption -- as opposed to post-processing/analysis on the receiving end. Here is an excellent thread on this topic: https://discourse.ros.org/t/pointcloud-compression-crowd-sourcing-benchmark/37648.

An alternative approach that works extremely well when you just want to "look at" the pointcloud, is to render it on the robot itself, e.g., using rviz running on a headless X display, and then streaming the video of that rendering over webrtc. In that case you can stream both an RGB image + the rendered pointcloud for less than 200KB/s and still get a very decent quality image. Even at 50KB/s you'd still find these streams usable, especially when the robot doesn't move around too much.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.