# Transformation matrix between a 2D and a 3D coordinate system

I have 2 sensors : one has 2D coordinate system and the other has a 3D coordinate system (not same origin).

Knowing the positions of some corresponding target points (used as fixed reference) in both coordinate systems,

How can I get the transformation matrix T (Rotation, translation and scaling) that would map the 3D point P3D to a 2D point P2D?
So that : P2D = T * P3D.

You need two things.

1. A transformation from 3D device coordinate to 2d device coordinate. $$p_{C}=T*p_{L}$$

2. Projective matrix
$$u_{C}=P*p_{C}$$ u is 2D location and p is 3D location.

To find 1 you need to do a 3D device to 2d device calibration. The method depends on your sensor configuration. Projective matrix can be found from your camera model.

• Thanks for the answer. then my problem now lies on the 1 (T matrix). What is the difference between Pc and Pl ? – Calips Nov 21 '18 at 14:39
• And it seems to me it's the same equation I gave already, because you could just write : uc = P * T * Pl ----> uc = M * Pl (If we take PL as the 3D point in input). – Calips Nov 21 '18 at 14:59
• Pc is the point in the camera coordinate Pl is the point in 3d device coordinate. T is 4x4 but P depends on your sensor configuration. To give you the correct answer in your situation you need to describe your system in detail. – C.O Park Nov 21 '18 at 16:12
• My setup is : RGB camera and An IR camera providing 3D positions of some reflective markers. What do you think ? – Calips Nov 22 '18 at 10:16
• In that case, I guess the marker points are in the IR camera coordinate. If you don't have the extrinsic T then you have to find it through a calibration first. P is a little bit tricky because you have to project and undistort the positions to be accurate. To do that you need a RGB camera intrinsic calibration as well. now your new equation for the second stage is different from the above. It is a vast topic. I recommend you to read about pinhole camera model, camera calibration. – C.O Park Nov 22 '18 at 11:36