I have a 5 axis robotic arm that has a depth camera mounted near the toolhead on the left side.

I have object detection software running on the camera and can find the XYZ of any object in mm that the camera sees relative to the middle of the camera.

The robot can move to any XYZ position in mm relative to the base mounting plate of the robot.

I'd like to get the arm to move to the position of an object seen by the camera. How would I go about doing this? Is there a software library for this?


1 Answer 1


The problem is solved with just a straightforward application of multiplying the transformation frames.


P: 3D point in camera frame

T_to_C: Transform from Camera to toolhead

T_B_to: Transform from toolhead to Base(your base mounting plate)

PB: Point in Base frame

Just multiply them through PB=T_B_to * T_to_C * P

You now have the XYZ of your Object in the Base Frame. Just calculate the inverse kinematics of that position to move your arm.


  • The transform between the toolhead and the camera is sometimes called the hand-eye calibration. If you don't have this then you must calculate it through a calibration process. ROS has a bunch of different packages for it.

  • Your transform between toolhead and base may be composed of multiple different transforms between the various joints in your robot arm.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.