Rosanswers logo


I run a Kinect sensor and evaluate depth and camera information to track positions of small robots. The idea is to use the pointcloud to identify all robots (2.5m < z < 3m). I used a pcl/passthrough filter for this purpose. Now I want to mask out all parts of the corresponding image which are not part of this cloud.

From my point of view I have to implement the following steps:

  1. Filter the point cloud (already available in pcl)
  2. Transfer the point cloud into an binary image (?)
  3. Mask the image (opencv?)

Other ideas? Thanks for your help.


Originally posted by Poseidonius on ROS Answers with karma: 427 on 2014-04-23

Post score: 1


1 Answer 1


Rosanswers logo


Ifound a solution for the problem and want to present them ... in order to motivate you for improvements:-) I used the

  • openni,
  • openni_launch,
  • depth_image_proc
  • pcl_ros
  • pcl_conversions

packages for ROS Hydro on Kubuntu 12.10. I used a ASUS Xtion for my setup.

1. Launch file

The solution combines 3 core elements:

  1. a nodelet, generating images and pointclouds based on the sensor
  2. a filter merging the 3D PointCloud and RGB Image information
  3. a filter that evaluates the depth values and adapts the RGB values
  4. a transormation that generates an image based on the PointCloud2.

The launch file illustrates the start up of these elements and follow at the end of the description part. I want to include some comments:

-> 1. The standard launch file $(find openni_launch)/launch/openni.launch has to be adapted related to the fact, that the following filter just accepts image and cloud data with the same frame id. I changed line 7 and 8 from the mentioned file

 <arg name="rgb_frame_id"   default="$(arg tf_prefix)/$(arg camera)_rgb_optical_frame" />
 <arg name="depth_frame_id" default="$(arg tf_prefix)/$(arg camera)_depth_optical_frame" />


 <arg name="rgb_frame_id"   default="$(arg tf_prefix)/$(arg camera)_optical_frame" />
 <arg name="depth_frame_id" default="$(arg tf_prefix)/$(arg camera)_optical_frame" />

in a new file myopenni.launch.

-> 2. The depth_image_proc package offers nodes for PointCloud2 processing. We use one of them for a fusion of point cloud data with corresponding RGB information. Take a view to the documentation on http://wiki.ros.org/depth_image_proc !

-> 3+4 The last two points are combined in the pointCloudToImage node. I wrote some lines of C++ to transform the ROS messages to PCL Pointcloud2 later to an pcl::PointCloudpcl::PointCloudXYZRGB object. It filters the RGB values according to the z coordinate. Afterwards the PointCloud2 is transformed to a ros image.

   #include <iostream>
   #include "ros/ros.h"
   #include <sensor_msgs/PointCloud2.h>
   #include <sensor_msgs/Image.h>
   #include <pcl/filters/passthrough.h>
   #include "pcl_conversions/pcl_conversions.h"

   #define Z_MIN 1.0
   #define Z_MAX 1.5

   ros::Publisher pub_cloud;
   ros::Publisher pub_image; 

   void cloudCallback(const sensor_msgs::PointCloud2 &ros_cloud)
     pcl::PCLPointCloud2 pcl_pc2;
     pcl_conversions::toPCL(ros_cloud, pcl_pc2); 
     pcl::PointCloud<pcl::PointXYZRGB>::Ptr pcl_cloud(new pcl::PointCloud<pcl::PointXYZRGB>);     
     pcl::fromPCLPointCloud2 (pcl_pc2, *pcl_cloud); 

     uint8_t r = 0;
     uint8_t g = 0;
     uint8_t b = 0;
     int32_t rgb = (r << 16) | (g << 8) | b; 

     for (size_t i = 0; i < pcl_cloud->points.size (); ++i)
       if ((pcl_cloud->points[i].z > Z_MAX) or (pcl_cloud->points[i].z < Z_MIN))
         pcl_cloud->points[i].rgb = rgb;
     pcl::toPCLPointCloud2 (*pcl_cloud, pcl_pc2);
     sensor_msgs::PointCloud2 ros_pc2;
     pcl_conversions::fromPCL(pcl_pc2, ros_pc2);

     pcl::PCLImage pcl_image;
     pcl::toPCLPointCloud2 (pcl_pc2, pcl_image);  
     sensor_msgs::Image ros_image; 
     pcl_conversions::fromPCL(pcl_image, ros_image);

   int main(int argc, char **argv)
     ros::init(argc, argv, "PointCloud2Filter");
     ros::NodeHandle n;
     ros::Subscriber sub = n.subscribe("input/", 1000, cloudCallback);
     pub_cloud = n.advertise<sensor_msgs::PointCloud2> ("output_cloud/", 30);
     pub_image = n.advertise<sensor_msgs::Image> ("output_image/", 30);
     return 0;

And the launch file ...

    <include file="$(find RoboMix)/launch/myopenni.launch" />
  <node pkg="nodelet" type="nodelet" name="pcl_manager" args="manager" output="screen" />
  <node pkg="nodelet" type="nodelet" name="fusion" args="load depth_image_proc/point_cloud_xyzrgb pcl_manager" output="screen">
    <remap from="rgb/camera_info" to="/camera/rgb/camera_info"/>
    <remap from="rgb/image_rect_color" to="/camera/rgb/image_rect_color"/>
    <remap from="depth_registered/image_rect" to="/camera/depth/image_rect"/>
    <remap from="depth_registered/points" to="/XYZRGBcloud"/>
  <node pkg="RoboMix" type="pointCloudToImage" name="ImageFilter">
     <remap from="input" to="/XYZRGBcloud" />
     <remap from="output_image" to="/resulting_image" />
  <node pkg="image_view" type="image_view" name="Visualisation" args="image:=/resulting_image" output="screen">

Feel free to improve !

Originally posted by Poseidonius with karma: 427 on 2014-05-05

This answer was ACCEPTED on the original site

Post score: 3


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.