Convert pointcloud 3d point to image pixel

Question

Hello there. I have a rosbag that contains a camera footage topic (sensor_msgs/Image) of a room and a laserscan topic of the same room. I also have a program that converts this laserscan to a pointcloud. What I'm trying to do, is write a program that gets as input a point in the pointcloud (x,y,z values) and finds the pixel coordinates of this point in the camera image (it's more important to me to find the horizontal coordinate though (width pixel) ). The specs of the camera are known, and so are the angle_min, angle_max, angle_increment, range_min, range_max of the laserscan. I'm a bit lost as to how I calculate this. Can anyone help me? Thanks in advance.

---

Originally posted by rbsb on ROS Answers with karma: 3 on 2018-07-30

Post score: 0

Answer 1

When you say the "The specs of the camera are known" what exactly is known?

This is a fairly straightforward transformation to do if you have the right information to hand. You'll need the lens calibration parameters of the camera, to be able to transform a 3D point in camera space into a 2D point in image space. You'll also need to know the extrinsic transform of the camera, in this case the relative pose of the LiDAR frame and the camera frame. This is used to transform the 3D LiDAR points into the camera frame.

You can determine the calibration parameters of the camera using this guide here.

If your LiDAR and camera are rigidly fixed together then you could measure by hand the extrinsic transform for a rough approximation. Note the orientation will need to be as precise as possible here. When you have this transform you can set it up as a static transform publisher following this tutorial, then ROS can easily transform the points into camera space for you. Note the different coordinate conventions of geometric and optical frames described here.

Finally you can use the openCV camera calibration module to do the final transformation from 3D points in camera space to 2D points in image space. The projectPoints function is what you'll use here.

Hope this gets you going in the right direction.

---

Originally posted by PeteBlackerThe3rd with karma: 9529 on 2018-07-30

This answer was ACCEPTED on the original site

Post score: 0

---

Original comments

Comment by rbsb on 2018-07-30:
Thank you very much for replying. The camera used is Orbbec Astra Pro. I should have also mentioned that I'm very new to ROS and robotics in general. I'm also having trouble understanding how the x,y,z coordinate system for pointclouds works (where is 0,0,0 for example?).

Comment by PeteBlackerThe3rd on 2018-07-30:
I have to say you've chosen a challenging task to start with if you're new to ROS. Have you tried any of the steps I've mentioned above? The coordinate system for the LiDAR point cloud will have its origin at the centre of the sensor itself.

Comment by rbsb on 2018-07-31:
Could you please elaborate on the "you could measure by hand the extrinsic transform for a rough approximation" part?

Comment by PeteBlackerThe3rd on 2018-07-31:
The exstrinsic transform between the LiDAR and the camera is the XYZ position of the camera in the LiDAR frame and the roll pitch and yaw of the camera in the LiDAR frame. If these sensors are rigidly attached to your robot then you should be able to roughly measure them by hand