I have 2 sensors : one has 2D coordinate system and the other has a 3D coordinate system (not same origin).

Knowing the positions of some corresponding target points (used as fixed reference) in both coordinate systems,

How can I get the transformation matrix T (Rotation, translation and scaling) that would map the 3D point P3D to a 2D point P2D?
So that : P2D = T * P3D.


You need two things.

  1. A transformation from 3D device coordinate to 2d device coordinate. $$p_{C}=T*p_{L}$$

  2. Projective matrix
    $$u_{C}=P*p_{C}$$ u is 2D location and p is 3D location.

To find 1 you need to do a 3D device to 2d device calibration. The method depends on your sensor configuration. Projective matrix can be found from your camera model.

  • $\begingroup$ Thanks for the answer. then my problem now lies on the 1 (T matrix). What is the difference between Pc and Pl ? $\endgroup$ – Calips Nov 21 '18 at 14:39
  • $\begingroup$ And it seems to me it's the same equation I gave already, because you could just write : uc = P * T * Pl ----> uc = M * Pl (If we take PL as the 3D point in input). $\endgroup$ – Calips Nov 21 '18 at 14:59
  • 1
    $\begingroup$ Pc is the point in the camera coordinate Pl is the point in 3d device coordinate. T is 4x4 but P depends on your sensor configuration. To give you the correct answer in your situation you need to describe your system in detail. $\endgroup$ – C.O Park Nov 21 '18 at 16:12
  • $\begingroup$ My setup is : RGB camera and An IR camera providing 3D positions of some reflective markers. What do you think ? $\endgroup$ – Calips Nov 22 '18 at 10:16
  • 1
    $\begingroup$ In that case, I guess the marker points are in the IR camera coordinate. If you don't have the extrinsic T then you have to find it through a calibration first. P is a little bit tricky because you have to project and undistort the positions to be accurate. To do that you need a RGB camera intrinsic calibration as well. now your new equation for the second stage is different from the above. It is a vast topic. I recommend you to read about pinhole camera model, camera calibration. $\endgroup$ – C.O Park Nov 22 '18 at 11:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.