I have one Picamera and one normal USB Webcam and would like to use them together as one stereo camera. Is this even possible as the 2 cameras aren't exactly the same? I would like to use them in ROS where I want to do SLAM. If it is possible, is there a guide on how to do it because as of now, I have only been using one camera for SLAM and it is not working too well.
$\begingroup$
$\endgroup$
3
-
$\begingroup$ In my opinion, as long u can perform chessboard calibration and get the intrinsic and extrinsic parameter of both camera, you should able to use them as stereo camera $\endgroup$– Albert H MApr 13, 2021 at 14:17
-
$\begingroup$ Note that both your cameras should be synchronized for the stereo algorithm to work correctly. This is not trivial. $\endgroup$– JuanchoApr 13, 2021 at 15:24
-
$\begingroup$ Wait, I am a little confused. I have done a chessboard calibration but what is this intrinsic and extrinsic parameter and how hard is this to set up. A guide or something would really help. $\endgroup$– DragonflyRoboticsApr 13, 2021 at 16:26
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
Yes, it is possible. That is often called a heterogeneous stereo camera.
Step1: stereo camera calibration.
Find relative camera location to each other (extrinsic).
Find camera lens distortion parameters (intrinsic).
Lens distortion is simply done by usual camera calibration but for the extrinsic finding, you need a bit of coding.
Step2: image rectifying
If you make the two cameras look in the same direction( same z-direction), you can make the processing a lot faster. But even if you make the hw well the z-direction can't be the same. Image rectifying makes the two images virtually parallel.
Step3: stereo matching.
Find a correspondence of a pixel on the left image in the right side image. The result is a disparity or depth map.
Step2 and 3 are where your problem is complicated due to the heterogeneous setup. It is likely you won't find an open-source code for the heterogeneous stereo camera. Even if you find one, it will be hard to use it without thoroughly understanding the stereo camera algorithm.
Camera synchronization is a big problem too especially if you need it for SLAM.
My recommendation is to try your best to void this configuration. Tell your supervisor it is semi-impossible and the expected outcome quality is way lower than a low-cost visual-inertial camera (I am sure this should be cheaper than your picam + usb cam configuration).
If you still would like to give it a try, see if Opencv supports a heterogeneous stereo camera setup.
-
$\begingroup$ So can you suggest a low cost visual-inertial camera? $\endgroup$ Apr 14, 2021 at 2:25