Relative scale in SfM

Question

I'm using monocular visual odometry. As in this case absolute scale can't be recovered, we rather compute relative scale for subsequent transformations as in this tutorial

To do this, triple matches across three frames are required. We then triangulate two $3D$ points $X_{m}$ and $X_{n}$ from images $\{i,i+1\}$ and $\{i-1,i\}$ then the relative scale can be determined from the distance ratio between point pairs in subsequent image pairs as follows:

$r=\frac{\parallel X_{m,\{i,i+1\}}-X_{n,\{i,i+1\}}\parallel}{\parallel X_{m,\{i-1,i\}}-X_{n,\{i-1,i\}}\parallel}$

My question concernes the coordinate system of the triangulated $3D$ points, the points have to be expressed in the coordinate system of the first, second or third frame or it dosen't matter as we are using the norm?

The triangulation method that I'm using outputs a $3D$ point expressed in the first viewpoint. How to convert it to another viewpoint?

Answer 1

The answer simply is, it does not really matter because you're using the norm. The scale is determined by the actual translation and rotation between two cameras (which in case of monocular odometry are two views from the same calibrated camera). This rotation and translation information is contained in the essential matrix (or fundamental matrix) which has a scale ambiguity.

Another point I'd like to mention is, triangulation using monocular images is seldom very accurate. So if your objective is to determine the scale of subsequent images to generate a trajectory, I suggest taking a median of many ratios between many corresponding points, because it will work much better than randomly selecting two points and calculating r.

To express your 3D point to another viewpoint, use bundle adjustment.