How does baseline work with forward motion in Monocular Visual Odometry

Question

In my computer vision class we are studying visual odometry. In this we calculate the Fundamental matrix using 2 frames from forward motion using the concept of stereo-vision. But in stereo vision the two cameras are side-by-side or at least not one after the another.

tldr, In VO we need a good enough baseline to reduce depth uncertainty. But how does that work when the frames are taken one after the another like in a moving car?

Answer 1

You are correct in your understanding that monocular VO will have an ambiguity in the scale of the reconstructed trajectory and scene.

The easiest way to fix this ambiguity is to incorporate more information via additional sensors. For example, a GPS measurement could give an absolute reference that can be used to scale the VO solution correctly. A barometer could give a better estimate of altitude changes, etc.

Alternatively, you could use the motion model of the platform to figure out the correct scale factor. In the case of a vehicle, the camera is mounted at a fixed distance and angle above the ground. In this case, your algorithm could be something along the lines of:

1. Compute your fundamental matrix (F) from two sequential frames
2. Compute the 3d points and rotation/translation between the sequential frames.
   • At this point, you will only have the points and translation up to scale, because there has been an unknown translation between the frames.
3. Use a sample/consensus algorithm, such as RANSAC, to find a ground plane in your computed 3d points
   • Other heuristics may be used to accelerate this (positions of points in the frame, etc)
4. Using your known physical configuration, compute the translation scale factor that makes the plane match.

For other platforms, you may be able to take advantage of the motion model to perform a similar "trick".