# Discover vector/angle between stereo camera pose and vehicle body

I have a calibrated stereo camera system that is mounted in a passenger car which means I am able to retrieve a point cloud from my stereo image. However, I need to find how well is the camera aligned with the vehicle - read: if the camera is perfectly facing forwards or not. I guess it will never perfectly face forwards so I need to get the angle (or rather 3D vector) between "perfect forwards" and "actual camera pose".

What came to my mind is to drive the vehicle possibly perfectly forwards and use stereo visual odometry to detect the angle of vehicle movement as seen by camera (which is the vector I am looking for). The LIBVISO library for visual odometry can output a 3D vector of movement change from one stereo frame to another which could be used to detect the needed vector.

The only problem may be to actually be able to drive perfectly forward with a car. Maybe an RTK GPS could be used to check for this or for correction. Will anyone have a suggestion on how to proceed?

The stereo camera I use consists of 2 separate Point Grey USB cameras. Each camera is mounted on a windshield inside the car with a mount like this one. The cameras were calibrated after mounting. The stereo baseline (distance between the cameras) is about 50 cm.

Sounds to me like you just need a one-shot test to check (stereo) camera alignment. I would just get a sheet of plywood screwed to a frame you can orient to the car. I would do this rather than trying to move the car because car micro motions are a pain.

So, with some crude ASCII art: O===O L

where the L is the plywood rig in front of your car. As long as you've taken care to make the rig square you can measure the distance between the edges and key features on the car. This ensures the plywood is square to the front of the car.

All of this is so you can get a point cloud of a flat sheet.

:EDIT:

I had originally said find the normal of the sheet, this is the direction the camera is pointing. After some thought last night, the normal of the sheet is going to be the mirror of the camera orientation.

Consider some more crude ASCII art. Always assuming the plywood is perfectly square and plumb and assuming the camera is pointing down and to the right:

\ |

What you will get from the camera's point of view is based on the camera's assumption that it is the one that is always pointed correctly, so it will represent the plywood in the scene with:

-- \

The orientation of the plywood as it rotates from the XY plane (assuming Z is distance from the cameras) is in the opposite direction the camera is rotated from straight ahead because these are two different aspects of the same rotation matrix.

That is to say, the camera will be rotated some amount relative to world coordinates (the car and thus the plywood), expressed by the rotation matrix $E$. The world (plywood) is rotated relative to the camera by $E^{-1}=E^T$

Cosine terms fall on the diagonal while sine terms are all skew symmetric off-diagonal, so transposing the rotation matrix is equivalent to using a negated rotation angle.

That is, the degree to which the normal of the plywood plane is rotated will be given by some rotation matrix $E'$, where $E'=E^{-1}$ and where $E$ is the rotation matrix of the camera. Using the inverse/transpose property of the rotation matrix, $E(\theta)^{-1}=E(\theta)^T=E(-\theta)=E'$.

Phew!

• I appreciate your answer but I am not really able to follow it almost at all :(. My stereo camera consists of 2 separate Point Grey cameras that are mounted inside a car on windscreen each with separate camera holder (like for GoPro cameras) - I added a link to the mount used. So the cameras are not mounted on a single plane, so I am not really sure how I should understand the plywood-thing. Also, what do you mean by micro motions in this case? – Kozuch Jun 14 '15 at 22:05
• Hold a flat sheet normal to the axis of the car. The flat sheet can be anything. Your camera assembly is attached to the car, pointed close to forward. If you can position the flat sheet perfectly normal to the axis of the car, you can measure it (generate a point cloud of it) with the stereo cameras. – Chuck Jun 14 '15 at 23:12
• If the Z direction is away from the camera assembly, then the flat sheet, as measured, should exist entirely in the XY plane, meaning every point will have varying X and Y coordinates but every point has the same Z coordinate. You can calculate the deviation from the XY plane by finding the normal of the measured sheet. – Chuck Jun 14 '15 at 23:12
• Now, I said flat sheet. I suggested a sheet of plywood because you can maneuver it easily. Put each side 6 inches from a headlight or bumper, then make sure it's plumb. Bam, it's aligned. You could use a wall, but what do you do if one side is 5.5 inches from the wall and the other is 6? Repark the car. Now one is 6 and the other is 6.1. This is what I mean by micro motion. – Chuck Jun 14 '15 at 23:14
• Fine positioning, which you need for calibration, is done well and typically easier by moving a target instead of the platform. As I said, walls are free and plentiful, but moving a car is hard and moving a wall is harder. So, to be clear, what I am suggesting is a stationary calibration target, not anything to do with your camera assembly. That can be anything, with any camera (or laser scanners, structured light, kinect, etc.) Let me know if you still have questions. – Chuck Jun 14 '15 at 23:19