2
$\begingroup$

Good day,

I am currently working on an obstacle avoiding UAV using stereo vision to obtain depth maps. I noticed that the quadcopter would sometimes not steer to the correct direction.

I am using the Raspberry Pi Compute Module IO board which comes with two CSI ports used with two v1 Pi Cameras.

Issue

I soon found out that due to the latency between the cameras, the left and the right images are not in sync thus the errors in the depth map result.

Steps taken:

I noticed the image blur when moving the cameras around so I adjusted the shutter speed by setting the UV4l/raspicam driver. With the shutter speed, I also tried to increase the framerate as I've read, it improves the latency issue. In my code which uses the opencv library, I used the grab() and retrieve() commands to replace the read() command so that the frames from both cameras is grabbed at the nearest time possible however it didn't help much.

Does anyone know any possible solutions?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You need real camera synchronization which means using cameras that both have external trigger capability (it is like the "remote shutter" on consumer/DSLRs). What you do is feed a common trigger signal to both cameras and sync them like that down to the sub-ms level (the accuracy largely depends on the camera HW but is usually very good, usually in us).

Frame synchronization is not implemented on the RPi camera so you can not sync two cameras for stereo (even though the v1 cam OmniVision sensor had a FREX pin which was kind of external trigger - but it was neither routed out of the sensor to the camera board nor implemented in the camera driver). What you are trying to do is is SW sync both raspi cams but both modules are in "free running" mode (they are streaming images at given FPS each continuously) and OpenCV only grabs the last frame that was taken which is far from being synchronous capture. Your frame offset may be as high as the frame rate is, but even at 90 FPS you still have some 10 ms difference which most likely is unacceptable for an UAV application.

Should you have a static environment you would not need to sync the cameras at all, but you seem to know this :). There were community attempts to frame sync the raspi cam, search for "frame synchronization" on the raspberrypi.org forum if you are interested.

If you ask for a low cost camera with external trigger - they cost about 300 USD from Point Grey - these are scientific USB cameras. The lowest cost I found is a See3CAM_10CUG from e-con Systems which starts at about $150.

Improve this answer
$\endgroup$
4
  • $\begingroup$ " they cost about 300 USD from Point Grey" Not really. The PS3 Eye camera has a hardware trigger (some soldering is required) so that it can be synchronized and costs around 10$. instructables.com/id/The-EyeWriter-20/step9/… $\endgroup$
    – FooTheBar
    Jul 15, 2016 at 6:46
  • $\begingroup$ I found a way to synchronize the cameras, UV4L has a stereoscopic option and outputs an image from the two cameras, one just needs to separate the two using simple Mat operations in opencv $\endgroup$
    – user123456098
    Jul 19, 2016 at 10:28
  • $\begingroup$ @user123456098: I doubt such UV4L stereoscopic capture will be synchronized in hardware - it most likely only joins the two last grabbed frames into one image... refer to my answer for the reasons why HW sync is not possible with RPi camera. $\endgroup$
    – Kozuch
    Jul 19, 2016 at 10:47
  • $\begingroup$ I am only running my stereovision algorithm at 4fps and the compute module is able to do that :) $\endgroup$
    – user123456098
    Jul 23, 2016 at 11:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.