Suppose I have two cameras $C_1$ and $C_2$ arranged in a stereo configuration. Let us assume that the camera is free of distortion with ideal intrinsics and that the camera extrinsics $R$ and $t$ are known. Let $C_1$ be the reference for the extrinsics of the system (i.e $R$ and $t$ are defined w.r.t to $C_1$). The two cameras have different focal lengths $f_{c_1}$ and $f_{c_2}$

stereo system

Now, in order to calculate disparity given the images $I_1$ and $I_2$ of the cameras $C_1$ and $C_2$, it is necessary that the images need to be rectified.

I have understood rectification as making the the images captured lie in the same plane in space. The baseline of the stereo $b$ is equal to the norm of the translation vector $t$. Let ($x_2$, $y_2$) be a pixel in the Image $I_2$ and ($x_1$, $y_1$) a pixel in $I_1$. I would like to rectify Image $I_1$ such that the it lies in the same plane as $I_2$.

Let $$P_1 = \begin{pmatrix}{x_1} \\{y_1} \\f_{c_1}\end{pmatrix}$$ and $$P_2 = \begin{pmatrix}{x_2} \\{y_2} \\f_{c_2}\end{pmatrix}$$ Would it be correct to say a pixel $P_2$ in $I_2$ is given by:

$$P_2 = \frac{f_{c_2}} {f_{c_1}}.(R.P_1 + t)$$

where $P_1$ is a pixel in $I_1$? We could then extend this and write $$ P_1 = R^T.(\frac{f_{c_1}} {f_{c_2}}P_2 - t)$$ and that $$P_{1_{rectified}} = R^T.(\frac{f_{c_1}} {f_{c_2}}P_2 - t) + t $$ since $t$ defines the baseline in the stereo system

Rectified system

Would the difference between $P_{1_{rectified}}$ and $P_2$ for all points $(x2, y2)$ in $P_2$ give me disparity? If not what would be the correct way to calculate disparity given the camera extrinsics and two cameras of different focal lengths. I have calculated the above expressions from my understanding of the geometry of the system. Please correct me if there are errors.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.