You are correct in your understanding that monocular VO will have an ambiguity in the scale of the reconstructed trajectory and scene.
The easiest way to fix this ambiguity is to incorporate more information via additional sensors. For example, a GPS measurement could give an absolute reference that can be used to scale the VO solution correctly. A barometer could give a better estimate of altitude changes, etc.
Alternatively, you could use the motion model of the platform to figure out the correct scale factor.
In the case of a vehicle, the camera is mounted at a fixed distance and angle above the ground. In this case, your algorithm could be something along the lines of:
- Compute your fundamental matrix (F) from two sequential frames
- Compute the 3d points and rotation/translation between the sequential frames.
- At this point, you will only have the points and translation up to scale, because there has been an unknown translation between the frames.
- Use a sample/consensus algorithm, such as RANSAC, to find a ground plane in your computed 3d points
- Other heuristics may be used to accelerate this (positions of points in the frame, etc)
- Using your known physical configuration, compute the translation scale factor that makes the plane match.
For other platforms, you may be able to take advantage of the motion model to perform a similar "trick".