I have a robotic arm 6-DOF and an Intel RealSense depth camera. I want to go from pixel values to 3D world coordinates, is it possible with eye-in-hand calibration? If yes can you guide me on how to perform this calibration? I have done some study on my own but have a lot of doubts.

  • $\begingroup$ What do you mean by pixel values? $\endgroup$
    – Ralff
    Mar 26 at 8:31

The steps are pretty easy to implement!

The Realsense provides you with the depth $d$ corresponding to the pixel $(u,v)$. Knowing from the forward kinematics of the robot arm the $4 \times 4$ homogeneous transformation $\mathbf{T}$ from the root reference frame to the 3D frame of the camera attached to the end-effector, the 3D coordinates of the point $\mathbf{p}$ projected in the pixel at hand are:

$$ \mathbf{p} = \mathbf{T} \cdot \left( \frac{d}{f} \cdot \left( u-\frac{w-1}{2} \right), \; \frac{d}{f} \cdot \left( v-\frac{h-1}{2} \right), \; d, \; 1 \right)', $$

where the resolution of the camera is $w \times h$ pixels, while $f$ is the camera focal length that you can retrieve from the Realsense API.

Clearly, $\mathbf{T}$ depends on the current configuration of the arm joints.

Bear in mind that the standard reference frame of the camera has the z-axis orthogonal to the image plane and pointing outward, the x-axis pointing rightward, and the y-axis pointing downward as specified by the pinhole camera model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.