0
$\begingroup$

Rosanswers logo

Hello,

I have two identical RGB cameras fixed to a rigid structure. They are synchronized by it's own node. Each camera is rotated clock-wise and counter clock-wise (due to where the usb connector is mounted on the camera's pcb). Therefore, I have rotated the images first and then publish them and use them for calibration and stereo_proc.

I've done the stereo calibration and managed to have the stereo_proc node working with help I got from this answer (mine as well, I didn't know if I should have opened a new question or not....but as it's a different topic I did).

I have successfully calibrated the cameras and obtained these left.yaml and right.yaml

left.yaml

image_width: 512
image_height: 512
camera_name: stereo/left
camera_matrix:
  rows: 3
  cols: 3
  data: [2344.331957, 0, 337.706086, 0, 2357.396758, 82.47229799999999, 0, 0, 1]
distortion_model: plumb_bob
distortion_coefficients:
  rows: 1
  cols: 5
  data: [-0.6336689999999999, 3.019293, 0.013087, 0.008579, 0]
rectification_matrix:
  rows: 3
  cols: 3
  data: [0.9948509999999999, -0.002312, 0.101323, 0.002147, 0.999996, 0.001734, -0.101327, -0.001507, 0.994852]
projection_matrix:
  rows: 3
  cols: 4
  data: [4243.110203, 0, 70.32520700000001, 0, 0, 4243.110203, 75.543232, 0, 0, 0, 1, 0]

right.yaml

image_width: 512
image_height: 512
camera_name: stereo/right
camera_matrix:
  rows: 3
  cols: 3
  data: [2346.048386, 0, 312.031898, 0, 2349.95312, 70.149203, 0, 0, 1]
distortion_model: plumb_bob
distortion_coefficients:
  rows: 1
  cols: 5
  data: [-0.604228, 1.382571, 0.015306, 0.004007, 0]
rectification_matrix:
  rows: 3
  cols: 3
  data: [0.993127, -0.003312, 0.116991, 0.003502, 0.9999929999999999, -0.00142, -0.116986, 0.00182, 0.9931319999999999]
projection_matrix:
  rows: 3
  cols: 4
  data: [4243.110203, 0, 70.32520700000001, -4319.654554, 0, 4243.110203, 75.543232, 0, 0, 0, 1, 0]

So with {left, right}/{image_raw, camera_info} I can obtain the rectified images and disparity map (even though not so good at the moment, therefore the other question) but that would concern the intrinsic parameters.

What about the extrinsic parameters? How do I extract them from camera_info and publish them to tf?. The stereo_calibration tutorial does not mention any of this...is it because I have to do something else? Such as use the industrial_extrinsic_cal node?

Thank you for the help.

EDIT 1: The extrinsic parameters that I mean are the rotation and translation from one camera to another as I want to do "pixel" triangulation (e.g: Specify in x,y,z (from left camera, as stereo_proc does with the pointcloud) the position of a pixel (u,v) in left image). So, i neet the relative position between the two cameras within the stereo pair as I'll know the transformation between base_link and the left camera in my robot.


Originally posted by Ariel on ROS Answers with karma: 65 on 2016-01-18

Post score: 1


Original comments

Comment by Dimitri Schachmann on 2016-01-18:
And please add whether you are interested in the extrinsic calibration with respect to some world frame, or between the two cameras.

$\endgroup$

1 Answer 1

0
$\begingroup$

Rosanswers logo

extrinsic with respect to what? sensor_msgs::CameraInfo does not contain any information about the extrinsic calibration (a.k.a. location with respect to some world frame) of the stereo pair. In ROS this information is stored only in tf.

When you publish your camera_info, it is up to you to also publish according tf information (if you need it), but note that for stereo cameras one usually needs only the reference frame of the left camera, which is then assumed for the whole stereo pair. That's how stereo_image_proc expects it to be, for example.


If you mean the relative position of the two cameras within the stereo pair, then it's implicitly encoded in the rectification matrix and the projection matrix. R and P in `CameraInfo.

By inspecting the implementation you will see, that all that is needed for stereo matching, is the rectification matrix (R) and the baseline (which is computed from P by -right_.Tx() / right_.fx(); where right_ is the CameraInfo from the right camera.)


Originally posted by Dimitri Schachmann with karma: 789 on 2016-01-18

This answer was ACCEPTED on the original site

Post score: 1


Original comments

Comment by Ariel on 2016-01-18:
So I can physically measure the baseline=10cm. Therefore, if I use the values from right P I should get the same number (4319.6/4243.1=1.01). Therefore, I can check that the calibration is not correct? BTW, right_.fx is also from P, right? Not from K? Shouldn't it be fx'() then?

Comment by Dimitri Schachmann on 2016-01-19:
I must admit I am currently unable to wrap my head around the math, but is StereoCameraModel::projectDisparityTo3d perhaps what you need? See in https://github.com/strawlab/vision_opencv/blob/master/image_geometry/src/stereo_camera_model.cpp

Comment by Ariel on 2016-01-19:
If I understood correctly, with projectDisparityTo3d I the left{u,v} pixel that I want the 3d coordinate, the value of that pixel (same u,v?) in the disparity image and I'll get the xyz point. Is that correct?

Comment by Dimitri Schachmann on 2016-01-19:
I must admit I'm a bit confused myself but yes: you give it pixel coordinates uv (of the left camera) and a disparity and it gives you 3D coordinates XYZ of that thing that you see at uv in the reference frame of the left camera.

Comment by Ariel on 2016-01-19:
What's making you a bit confused? I'll give it a try a check how things work....but based on my first comment here, there is something wrong with the calibration as the baseline gives me 1.01 meters rather than 0.1 meters.....

Comment by Ariel on 2016-01-20:
One thing that I still don't understand is the double disparity variable. I have three values for each pixel in the disparity topic (R,G,B) and I need one value for the projectDisparityTo3d function. How does that work?

Comment by Dimitri Schachmann on 2016-01-20:
Disparity images must have only one channel! It makes no sense for them to have three channels. Where do you get your stereo_msgs/DisparityImage from? What does msg.image.encoding contain?

Comment by Ariel on 2016-01-20:
My mistake....the encoding is 32FC1. I still cannot obtain a decent disparity image.....When I use max resolution (1280x1024) rather than 30fps that I get from the raw data from the cameras, the disparity images is ~1fps....could it be some bottleneck in stereo_image_proc?

Comment by Dimitri Schachmann on 2016-01-20:
Do the disparity images, even though 1fps, look sensible? If not, then your calibration is most probably wrong. What does htop say about your CPU usage, how much is stereo_image_proc taking up?

Comment by Dimitri Schachmann on 2016-01-20:
I also suggest the following cheap check: display left/image_rect and right/image_rect side by side and check whether they look rectified as one would expect. (i.e. same features on same pixel row and in the right image further to the left).

Comment by Ariel on 2016-01-20:
The disparity images (in that example is with smaller resolution and calibrated at that resolution) looks like this.I'm still not sure about whether rotating the images is correct for calibration,but if I don't do it I get black rect'd imgs

Comment by Ariel on 2016-01-20:
Here is a screenshot of the disparity image at max res (with proper calibration) and htop on the side.

Comment by Dimitri Schachmann on 2016-01-20:
Well can not see anything wrong in the rectified images, but the disparity image is completely wrong. Those specs are just accidental matches. Why and how do you rotate the images? And yes stereo_image_proc gives you a lot of load on the CPU, but I'm surprised that it's that much!

Comment by Ariel on 2016-01-20:
I tested again and changed the disparity_range to a much bigger value and I started getting something different as shown here but there is still a lot of grey areas. The images seem to be well rectified.

Comment by Ariel on 2016-01-20:
Here you can see at the top the raw images (cameras are fixed and rotated due to their usb connector...). At the bottom are the rotated images using cv::transpose and cv::flip(other node) I rotated the image because the rectified images were completely black

Comment by Dimitri Schachmann on 2016-01-20:
mmh. I'm afraid I'm out of ideas for now...

Comment by Ariel on 2016-01-20:
Would it be worth it to implement the opencv calibration myself? Would I have different results? This way I could also have all the extrinsic parameters....but would I get dif result than with ros calib?

Comment by Ariel on 2016-01-21:
I checked that I put the wrong value for the square size when calibrating. I fixed it, didn't rotate the images and these are the rectified images. The image is much smaller. The disparity is still a mess.

Comment by Dimitri Schachmann on 2016-01-21:
your image is quite homogeneous, try to hang something with "features" on the wall.

Comment by Ariel on 2016-01-22:
I changed sides and now I have more things in the scene: rectified disparity rviz. There is something in the disparity but in rviz I cannot see anything....

Comment by Ariel on 2016-01-22:
I have just re-done the calibration and checked the epi value afterwards and it averages 0.45.......this should be way smaller, right?

Comment by Dimitri Schachmann on 2016-01-22:
perhaps you could record a (very) short bagfile with your data and upload it somewhere?

Comment by Ariel on 2016-01-22:
which topics? image_raw, camera_info? here it is

Comment by Ariel on 2016-01-25:
I checked the code and even though translation vector and rotation matrix are not saved, they are printed on the terminal when the calibration is done. These are the R and T from cv::stereoCalibrate. I still cannot get a good disparity image...

Comment by ducnan on 2020-02-11:
So I am also trying to get the baseline of the stereo camera. Which one is the right._Tx()? Is it the first element in the self.T matrix? And as for the right.fx(), is it the first element of the projection matrix for the right camera?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.