# Understanding types of hand eye calibration

I have been reading about hand eye calibration for a couple days. Basically, I have 2 setups I want to execute hand eye calibration.

1. Camera is fixed in a position and arm is moving (eye on base);
2. Camera is fixed to the robot wrist (eye on hand).

I have read Tsai's paper and a lot of other documents and read some similar question posted here (Hand Eye Calibration)

I also, have found some libraries that I can achieve this with ROS.

1. easy_handeye: https://github.com/IFL-CAMP/easy_handeye
2. handeye_calib_camodocal: https://github.com/jhu-lcsr/handeye_calib_camodocal

I believe that both hand-eye setups I described before can be solved with formulation AX=XB (since easy_handeye solves this way). Furthermore, I believe that setup 2 (eye-on-hand) is the classic problem for AX=XB. However, I cannot understand how to model the problem in order to solve for setup 1 (eye-on-base). Also, I cannot understand the difference between AX=XB and AX=ZB (which sometimes is written as AX=YB).

This is a more theoretical question, but I like to understand the theory before start using any tool, otherwise I will have some technical debt that can generate problems in the future. If someone can explain to me or provide material to clarify these points, I will be very grateful.

• No need to apologise, this is a great first question. – Mark Booth Apr 14 '20 at 7:38

The references you gave actually have all the information that you need. I think you are maybe getting confused a bit with the different coordinate systems. As the same symbol can have subtle different meanings.

difference between AX=XB and AX=ZB

If you write the transforms with the coordinate subscripts then it becomes more clear. The equations then become:

$$A_{12}X=XB_{12}$$ $$A_{wc}X=Z_{w}B_{wh}$$

• $$w$$ is world origin
• $$c$$ is camera frame
• $$h$$ is hand frame

So $$AX=XB$$ is used when you have delta transforms. You have the camera on the wrist(scenario 2). You move the robot arm around to estimate $$X$$, which is the sole quantity you are trying to estimate. Note you don't ever actually care about any absolute poses. You only care about the transforms between 2 poses. If you look at the image you can see that $$A$$ and $$B$$ are between different poses. Not from some origin.

$$AX=ZB$$ is used when you have or are interested in the poses with respect to some coordinate frame(e.g World). You then need an extra term to represent this transform which is $$Z$$. Now $$Z$$ might be given to you are it might be something you have to simultaneously estimate. It depends on your specific problem.

The images in the stackoverflow post you link to show this quite well. The parameters with the question marks are the ones you are trying to estimate.

1 (eye-on-base)

It is a bit different then the general $$AX=XB$$ or $$AX=ZB$$ as here $$X$$ is not constant. Instead it is the value that changes and we are interested in estimating $$A$$.

The equation should be something like this

$$A_{bc}X_{ch}=B_{bh}$$

• $$b$$ is the robot base

and could be modified to look like $$AX=ZB$$ by adding a world coordinate frame.

$$A_{wc}X_{ch}=Z_{wb}B_{bh}$$

which corresponds to the picture.

To solve this you put a marker on your robot hand so the camera can track it as seen in your link here. $$Z_{wb}$$ is arbitrary and depends on what you consider your origin. $$B_{bh}$$ comes from your odometry/encoders, and $$X_{ch}$$ comes from the marker tracking. You can then solve for $$A_{wc}$$

• Thanks for the explanation edwinem, It really clarified for me. I was and still confused with the coordinate systems. Mathematically it is very clear now. – Gabriel Carvalho Apr 10 '20 at 3:17
• However, I am still a bit confused on how to use the available tools, for example, VISP has a hand eye calibration method vpHandEyeCalibration::calibrate which is used by easy_handeye. – Gabriel Carvalho Apr 10 '20 at 3:28
• In their code, they mention a "trick" to be able to use such library to calibrate in the eye-on-base situation, and the only difference is that instead of estimating the transformation from base to effector (eye-on-hand) they estimate the transformation from end effector to base (eye-on-base) then the transformation from camera to marker is estimated normally in both cases. – Gabriel Carvalho Apr 10 '20 at 3:29
• Given these details I understood that the calibrate method receives two series of transformation matrices (inputs) and outputs a third transformation matrix that related the inputs, and the developer has to be careful on which relationships he inputs to the algorithm in order to get the correct relationship back. – Gabriel Carvalho Apr 10 '20 at 3:30
• I don't get what exactly you are asking for. Are you asking how to use the software as that seems relatively straightforward as seen in the README file for easy-handeye. The trick makes sense. I recommend drawing the frames on paper and labeling all the transforms. It works by making the eye-on-base transform the $X$ part of the equation which makes sense as it is constant and doesn't change. – edwinem Apr 10 '20 at 6:11