Dealing with transparent objects, combining two Xtion

Question

Hello,

I'm trying to make 3D map of my office with XtionPRO Live, the biggest problem i met so far is dealing with transparent/aborbing objects etc. I tried to avoid them but they still reflects (windows, glass doors) and i got random erros on my map (like inclined in the middle of room). Other things like glasses or bottles are hard to avoid. I couldn't find any solution and it's seems to be impossible to solve problem due to Xtion sensor is using IR.
The only thing i found is publication about combining two Xtions (http://www.cs.cornell.edu/~hema/rgbd-workshop-2014/papers/Alhwarin.pdf). I'm new to RoS/mapping and publication I found is too complex for me to implementation. I understand main assumption but I'm not sure how to prepare it and use with RTAB-map.
I am able to get second Xtion/Kinect but how should i calibrate them? Earlier i used (http://wiki.ros.org/action/show/openni_launch/Tutorials/IntrinsicCalibration) but in that way i can pass mono calibration for sensors separately, how can i combine them? Is there any way to achieve goal with available packages? Is there any other solutions to deal with objects like that in RTAB-map?

Thanks for answers.

---

Originally posted by necro on ROS Answers with karma: 38 on 2016-07-25

Post score: 0

---

Original comments

Comment by Mehdi. on 2016-07-25:
I don't to which extent you can integrate it in RTAB map but sonars are the solution to transparent objects.

Answer 1

Reflective and transparent surfaces are very challenging for mapping approaches. For the two Xtions approach you refer, it is a stereo setup with IR images. You may use stereo calibration tutorial to calibrate the Xions with each other. Well, you may have to hide the IR pattern when calibrating. On runtime, you may use stereo_image_proc to generate the disparity image. Be aware that this package may assume horizontal stereo, so you may have to rotate the images/camera_info, then rotate back the output disparity. By converting the disparity to depth (rtabmap_ros/disparity_to_depth) and left IR image to Gray scale, you could use rgb-d interface of rtabmap.

However, some surfaces are just very difficult to detect with vision/laser, just think about a mirror or clean glass...

---

Originally posted by matlabbe with karma: 6409 on 2016-07-26

This answer was ACCEPTED on the original site

Post score: 2

---

Original comments

Comment by necro on 2016-07-26:
Thanks for Your involvement! Already I'm waiting for second Xtion and in meantime I got ZED camera to play with. ZED Wrapper got this file, so why i need to run rosrun tf static_transform_publisher? Seems very similar for me what's the diff

Comment by matlabbe on 2016-07-26:
Oh, that commit (June 21) modified the TF frames of the ZED. They are also starting their own odometry with the camera driver. The tutorial should be updated.

Comment by necro on 2016-07-26:
So what should i do? Without line from Your tutorial i got:

odometry: Could not get transform from camera_link to /zed_tracked_frame (stamp=1469541052.943110) after 0.200000 seconds ("wait_for_transform_duration"=0.200000)!

After running your line i can successfully see rtabmap gui.

Comment by necro on 2016-07-26:
Other thing is that my clouds are "waved". My ZED is uncalibrated in ROS but i read that it's pre-calibrated and I don't need to do anything. Here is my screenshot:
http://i.imgur.com/Q6Fz9on.png
The wall on right isn't straight. Does that camera need more clouds to give depth?

Comment by necro on 2016-07-27:
Using ZED Depth Viewer with this tool I could obtain pointclouds straight from the camera and it's waved too. Other thing is that I can see only one cloud, because tool ZEDfu for mapping is still unreleased for Linux. I will try to check it under Windows and see if assembling will straighten image.

Comment by matlabbe on 2016-07-27:
You may set a maximum cloud depth to filter far waved clouds. See also "To have similar results than ZedFu..." section in https://github.com/introlab/rtabmap/releases/0.11.7.