# Effect of camera misalignment errors in dual cameras

In the camera module assembly process, parameters of camera modules vary due to manufacturing tolerance. Camera calibration is performed to obtain actual parameters. In the paper Effects of camera alignment errors on stereoscopic depth estimates, the author analyzes relative sensitivity/importance of camera calibration/alignment parameters on the performance of stereoscopic depth reconstruction.

For dual camera system, five sources of error are listed

Binocular error effects:

1. depth error due to rotation/roll between two cameras
2. depth error due to pitch between two cameras
3. depth error due to yaw between two cameras

Monocular error effects:

1. depth error due to nonparallel CCD array and lens
2. depth error due to lens distortion.

In practical applications, camera rectification will use these parameters to align two images. Why do we need to analyze the effect of various errors?

## 1 Answer

Because each have diferent reasons (and so can be helped/worsened in different ways).

And the calibration will help only in specific setup and needs be done differently for different setups.

The easiest way to understand is if you take two cameras, whitch are ideal, just not parallel - then you get the image of the same on different parts of resulting image and it would be worse with distance, so you can set it up perfectly for one distance and aproximate that for all other distances.

On the other hand if your cameras are ideal, just their distance is not perfect, you get the same image as in previous case, but this time the solution is to just substract the missalignemnt from the position and the error would became less with distance.

But if your cameras are rotated, you need to transform the whole picture of one of them to align the item on both - again different problem and solution.

The lens distortion can be the same for full image, or can differ in different parts - mainly in corners - so if your object of interest would not be in centre, you have another complicated problem to solve.

So if you are to use it at given angle and distance, you calibration can be a lot simpler, then if you want to use it on wide range of distances and angles to measured object.

And if you want to write your own software/dirvers/addons, you need to know all those sources of errors, as you may need to include them (and their respective corrections) in your software. And it is better to measure them yourself with your piece of HW, than to rely on average tolerance given by manufacturer for the whole production line.

Also the corrections can be done fast (and somewhat inaccurate) with a little space needed, or slow with a lot of sources consumed but giving better precision - even if manufacturer provides some correction method, it is necessarry compromise to somehow fit needs of mayority of perceived customers while not taking too much time and sources.

But you particular application may have different criteria and would like to have at least some results really fast (racing car), or need as precise results as possible (scientific observation), or needs just something with less work needed (hobby project), so it is good to mention, what sources of error are there and how they differs, so you can choose the approach optimal for you.

• Thanks for your valuable answer. For one dual camera system, I use stereo calibration to obtain the information(R/T) about geometrical relation between them and intrinsic parameters for each camera. Even if two optical axes are not parallel, camera rectification process will make them parallel. The two cameras will become perfect after rectification. Why do I need to care about sources of error. May 2 '17 at 15:12
• That is just about what you need. Somebody does not care and is happy, somebody wants tu push limits and solves everything possible. It is better to have more information, than have some info missing. (on some not so related note - I got "library" for Arduino for 4 digit display, comments say "works just fine, fully satisfied" - refreshed like 10x per sec., some digits was brighter, all had little "ghosts" on dark parts, occupying Arduino to full time. I rewrote it like 4x, now it refreshes at 1kHz, no ghost, CPU mostly free. Just needed to find tech specs. I still see space for improvements.) May 2 '17 at 21:41
• I hope to obtain as possible as good result. So I read some paper to try to understand the principle behind it. I still can't fully understand your answer. Do you mean calibration procedure can obtain actual parameters. The parameters may include error? The rectification can only correct minor error? May 2 '17 at 23:27
• In stereo camera calibration, reprojection error is obtained which is the combined effect due to different error sources. How can I analyze the error? Maybe it is another question about the accuracy of the calibration process. In the camera module assembly, a lot of equipment is used to make optical axes of two cameras parallel. Is it necessary? Can camera calibration and rectification handle manufacturing variation? May 3 '17 at 5:16