If my understanding is correct, one flavor of hand eye calibration requires the following pairs of data for different robot arm positions:
- The transform from the robot base to the robot hand.
- The transform from the marker to the robot camera.
By transform, I mean the 4x4 homogeneous matrix that contains both rotation and translation information.
The part I am stuck on is getting the transform from the marker to the robot camera, which is a 2D camera. The marker can be checkerboard, or Aruco, etc. for which opencv can give you the pose of the marker with respect to the camera.
The question I have is fundamental: Is it even possible to get the pose of the camera from images of a marker of known size? Won't there be a scale ambiguity specifically in the z direction of the camera? Is this z ambiguity resolved by using a marker of known size? Finally, if this scale ambiguity is always there, is it possible to do hand eye calibration using a 2D camera and a marker of known size?