# Stereo vision on a moving vehicle

When I put stereo camera on a moving platform and sync the cameras, what kind of delay between the cameras is tolerable? This would probably depend on the motion, but are there some guidelines?

Are there any practitioner guides on how to build a moving stereo system?

• From my experience I can say: This depends on the algorythm! The simplest algorythms are not very tolerant to such things. later algorythms can handle some small delays (the quality will decrease for sure) I suggest take the images at the same time and minimize the delay. Everything under 100ms is no problem at all Commented Nov 13, 2014 at 10:28
• Why do you say that under 100ms is not a problem? For what speeds? I have a hardware synced stereo pair which produces about 100 ms delays. Bouguet stereo calibration toolbox refuses to calibrate the pair if the calibration patters move, so 100ms for calibration is probably not acceptable... Commented Nov 13, 2014 at 10:39
• Do you use a "real" stereo camera or 2 independent cams? I worked with 2 indepentend cams at a framerate of 10hz. This cameras aren't synced, so the picture could be up to 100ms old and the other picture may be a pretty recent one So there is a delay in the pictures. Possible workaround for you: take pictures at 10hz and delay one by one cycle. then you eliminated the hardware delay Commented Nov 13, 2014 at 10:47
• I use 2 separate cams. The first camera is software triggered, but when its sensor is exposed it hardware triggers the 2nd camera. Commented Nov 13, 2014 at 10:56
• Ok the delay is exactly 100ms? Commented Nov 13, 2014 at 11:37

The guidelines is quite simple math. You are right the sync precision requirement is a function of speed - the faster you go the better the sync has to be to give you the same distance difference between the two cameras. Example: 100 km/h is 27.7 m/s - 100ms sync will give you difference of 2.7 meters of distance from one camera to other. 10ms will give you 27cm, 1ms 2.7cm etc. If you have a very slow moving vehicle 100ms can be OK. But I think no stereo algorithm (3D reconstruction, visual odometry) can produce good results with bad input data so you want to do your math before starting to see where the sync should be.

If you can analyze your stereo in post you can fix the sync in post-processing (more FPS will give you better sync possibility). If you need real-time then you can not fix in post and your requirement for hardware is higher.

For real time stereo processing I think you want the stereo sync to be as good as possible within your budget (of course depending on vehicle speed). The best is two identical cameras with wired external trigger (remote shutter) - you wire both cameras to the same HW trigger button. If you dont need high FPS you can use commodity (relatively cheap) consumer cameras with remote shutter (few point&shoot cameras may have it, ultrazooms or DSLRs will too), but these usually save to SD card which is not very convenient for real-time processing.

An USB camera is better for real time as you can stream the images directly into your code (C++ etc.). "Industrial" cameras are usually USB, they have quite high FPS and external trigger - See3CAM_10CUG by e-con Systems is the cheapest one probably, other more expensive brands are The Imaging Source or Point Grey. These can usually do from about 10 to many FPS with external trigger depending on model.

I use 2 separate cams. The first camera is software triggered, but when its sensor is exposed it hardware triggers the 2nd camera.

This is a wrong solution - you want to trigger both the same way in order for them to behave the same - see the remote shutter above.

EDIT:

Going back to the root of your question:

what kind of delay between the cameras is tolerable?

As I said above this translates into "what distance difference between left and right stereo frames is tolerable for stereo vision?" To be honest, even though I did some stereo matching I have no experience with sync precision here (I used synced dastasets), but as I said I can imagine the results will get worse linearly with the distance delta. Would be interesting to do some tests in this field...

Something like "spherical resolution" (number of image horizontal/vertical pixels divided by your lens field-of-view) will probably play a big role - the lower the spherical resolution the lower the sync precision needs because you could not recognize the details anyways. There will be some (probably not very difficult) math behind this.