Discover vector/angle between stereo camera pose and vehicle body

Question

I have a calibrated stereo camera system that is mounted in a passenger car which means I am able to retrieve a point cloud from my stereo image. However, I need to find how well is the camera aligned with the vehicle - read: if the camera is perfectly facing forwards or not. I guess it will never perfectly face forwards so I need to get the angle (or rather 3D vector) between "perfect forwards" and "actual camera pose".

What came to my mind is to drive the vehicle possibly perfectly forwards and use stereo visual odometry to detect the angle of vehicle movement as seen by camera (which is the vector I am looking for). The LIBVISO library for visual odometry can output a 3D vector of movement change from one stereo frame to another which could be used to detect the needed vector.

The only problem may be to actually be able to drive perfectly forward with a car. Maybe an RTK GPS could be used to check for this or for correction. Will anyone have a suggestion on how to proceed?

The stereo camera I use consists of 2 separate Point Grey USB cameras. Each camera is mounted on a windshield inside the car with a mount like this one. The cameras were calibrated after mounting. The stereo baseline (distance between the cameras) is about 50 cm.

Answer 1

Sounds to me like you just need a one-shot test to check (stereo) camera alignment. I would just get a sheet of plywood screwed to a frame you can orient to the car. I would do this rather than trying to move the car because car micro motions are a pain.

So, with some crude ASCII art: O===O L

where the L is the plywood rig in front of your car. As long as you've taken care to make the rig square you can measure the distance between the edges and key features on the car. This ensures the plywood is square to the front of the car.

All of this is so you can get a point cloud of a flat sheet.

:EDIT:

I had originally said find the normal of the sheet, this is the direction the camera is pointing. After some thought last night, the normal of the sheet is going to be the mirror of the camera orientation.

Consider some more crude ASCII art. Always assuming the plywood is perfectly square and plumb and assuming the camera is pointing down and to the right:

\ |

What you will get from the camera's point of view is based on the camera's assumption that it is the one that is always pointed correctly, so it will represent the plywood in the scene with:

-- \

The orientation of the plywood as it rotates from the XY plane (assuming Z is distance from the cameras) is in the opposite direction the camera is rotated from straight ahead because these are two different aspects of the same rotation matrix.

That is to say, the camera will be rotated some amount relative to world coordinates (the car and thus the plywood), expressed by the rotation matrix $E$. The world (plywood) is rotated relative to the camera by $E^{-1}=E^T$

Cosine terms fall on the diagonal while sine terms are all skew symmetric off-diagonal, so transposing the rotation matrix is equivalent to using a negated rotation angle.

That is, the degree to which the normal of the plywood plane is rotated will be given by some rotation matrix $E'$, where $E'=E^{-1}$ and where $E$ is the rotation matrix of the camera. Using the inverse/transpose property of the rotation matrix, $E(\theta)^{-1}=E(\theta)^T=E(-\theta)=E'$.

Phew!