I am using a PNP algorithm to compute the rotation and translation of a camera given pre-mapped 3D points, and their corresponding projections on the 2D plane. The algorithm I am using is the one described in a CVPR 2017 paper, with implementations in both OpenCV and OpenMVG (I am using the latter).

Strangely, I am noticing a precise 'drift' in the position computed, that changes with the rotation angles. I.e.: If I hold the camera's position constant and just rotate it, according to the OpenCV coordinate convention, changes in pitch cause the position to drift in the Y direction, and changes in yaw cause the position to drift in the X direction (I have not tested roll yet.) When I tried to fit a curve to this dataset of pitch vs Y and yaw vs X values, I noticed a pretty constant variation. After removing scale factors and converting the angles into radians, this is the profile I see:

\begin{eqnarray} X \approx -4.0 * \psi \\ Y \approx 4.0 * \theta \end{eqnarray}

My translation units in this equation are meters but not true world units, because I removed the scale factor; angles are in radians. In true world units for the environment I tested in, the factor of 4.0 ended up being a factor of 20.0. Is this some sort of a commonly known transformation I am failing to account for? It's linear, so I am assuming it cannot be dependent on the rotation angle. I have skimmed through the paper and some relevant resources but I am unable to figure out the cause of this relationship.

Curve fitting results from MATLAB: Pitch curve Yaw curve


The problem was a bug in my code where I was accessing the translation part of the solution directly, and not the camera position, so it was indeed a missing transformation.

For future reference, the true camera position from a PNP solution needs to be computed as

\begin{equation} C = -\mathbf{R}^\top * t \end{equation}

$\mathbf{R}$ is the rotation matrix and $t$ is the final translation vector returned by the PNP algorithm. Being a linear operator, this encodes a purely linear relationship with respect to the rotation angles.

  • 1
    $\begingroup$ Glad you found the answer! You can accept your own answer with the check mark to the left of the answer text, but you might need to wait a day or two to do so. I'm curious - was it a naming problem (as in, you accessed translation instead of camera) or were you supposed to have applied the transform above (as in, there is no true camera position available in the code)? Also, what is $t$ in your formula? Congrats on finding the bug! $\endgroup$
    – Chuck
    Jan 24 '18 at 14:52
  • $\begingroup$ It was indeed a naming problem: I was already applying this transformation and storing it as pose.position() in my structure, but I made the mistake of accessing pose.translation() when logging. $\endgroup$ Jan 24 '18 at 14:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.