2
$\begingroup$

Suppose I have two cameras $C_1$ and $C_2$ arranged in a stereo configuration. Let us assume that the camera is free of distortion with ideal intrinsics and that the camera extrinsics $R$ and $t$ are known. Let $C_1$ be the reference for the extrinsics of the system (i.e $R$ and $t$ are defined w.r.t to $C_1$). The two cameras have different focal lengths $f_{c_1}$ and $f_{c_2}$

stereo system

Now, in order to calculate disparity given the images $I_1$ and $I_2$ of the cameras $C_1$ and $C_2$, it is necessary that the images need to be rectified.

I have understood rectification as making the the images captured lie in the same plane in space. The baseline of the stereo $b$ is equal to the norm of the translation vector $t$. Let ($x_2$, $y_2$) be a pixel in the Image $I_2$ and ($x_1$, $y_1$) a pixel in $I_1$. I would like to rectify Image $I_1$ such that the it lies in the same plane as $I_2$.

Let $$P_1 = \begin{pmatrix}{x_1} \\{y_1} \\f_{c_1}\end{pmatrix}$$ and $$P_2 = \begin{pmatrix}{x_2} \\{y_2} \\f_{c_2}\end{pmatrix}$$ Would it be correct to say a pixel $P_2$ in $I_2$ is given by:

$$P_2 = \frac{f_{c_2}} {f_{c_1}}.(R.P_1 + t)$$

where $P_1$ is a pixel in $I_1$? We could then extend this and write $$ P_1 = R^T.(\frac{f_{c_1}} {f_{c_2}}P_2 - t)$$ and that $$P_{1_{rectified}} = R^T.(\frac{f_{c_1}} {f_{c_2}}P_2 - t) + t $$ since $t$ defines the baseline in the stereo system

Rectified system

Would the difference between $P_{1_{rectified}}$ and $P_2$ for all points $(x2, y2)$ in $P_2$ give me disparity? If not what would be the correct way to calculate disparity given the camera extrinsics and two cameras of different focal lengths. I have calculated the above expressions from my understanding of the geometry of the system. Please correct me if there are errors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.