How to get transform between two fixed frames using a camera

In my task, I have two manipulators: one 6 dof robot (Manipulator 2 as shown in attached link picture), and one passive arm (Manipulator 1 as shown in attached link picture). Both the frames 1 and 2 are fixed. I would like to transfer the movement that I have with respect to the frame 1 to the frame 2, for this I need to compute the transform between the two fixed frames. To this end I would like to use a camera in a third frame and fiducials on the arms for defining the frames, how can I get the frames axes using the camera? Thank you in advance for any helpful insights!

• Welcome to Robotics, Lisa9892. I'm afraid I don't understand your question here - you've provided a drawing but there's no camera. Is the camera mounted to one frame (or robot) and you're trying to estimate the other? Is it in some third frame location and you're trying to estimate the difference between the two? I also don't know what you mean when you manipulator 1 is a passive arm. Is there any advance knowledge of the two frames or are you trying to fix the transform between the two arms in six degrees of freedom? May 16 at 13:24
• Hi Chuck, thank you very much for your reply. the camera will be located on a third frame and what I want to compute is to get the transform from the frame 2 to the camera and then from the camera to the frame one. To this end I would like to define those two fixed frames (or axes) with less errors. I mean by the manipulator 1 is a passive arm that he doesn't include actuators on its joints. I hope I could clarify. Thanks! -Lisa May 17 at 7:12
• I am also opened for any other insights! May 17 at 7:14
• What are you using for the camera to be able to recognize the frames? Do you have some fiducials like ARTags on the arms or bases? Or are you trying to recognize the arms themselves?
– Ben
May 18 at 16:08
• On stack exchange, it is better to edit your question to add information requested in comments, rather than adding more comments. Comments are for helping to improve questions and answers, and are distracting, so we try to keep them to a minimum. If all of the information needed to answer the question is contained within it, the comments can be tidied up (deleted).
– Ben
May 18 at 16:09

The most simple and easy way is using a common third frame.

1. Attach a camera to the end effector of each robot and place a calibration board on the ground.
2. Do a hand to eye calibration using the calibration board
3. Find the base frame transformation matrix from the calibration board
4. Now you can calculate Frame 1 to Frame 2 ($$^{F1}T_{F2}$$) as follows

$$^{F1}T_{F2} = ^{F1}T_{F1End} * ^{F1End}T_{F1Cam} * ^{F1Cam}T_{Calib} * (^{F2}T_{F2End} * ^{F2End}T_{F2Cam} * ^{F2Cam}T_{Calib} )^{-1}$$

There could be multiple variants of this method.

• Hi Chanoh, thank you for your reply. Could you please explain what do you mean by a third common frame? willI have to use only one camera or two? Thanks May 20 at 11:40
• Put a calibration board on the floor which will be your common frame. F1 to calibration board and F2 to calibration board can be found. Now you can find F1 to F2. Solution 1: one camera somewhere and two calibration board at robot TCP. Solution 2: two cameras at each TCP and one calibration board on the floor. May 22 at 11:51
• Thanks Chanoh! do you have any recommendation regarding implementation ? via Matlab or OpenCv? May 23 at 17:54
• Matlab is enough for the proof of concept. Use opencv, eigen, ceres if you want to implement it in c++. May 25 at 0:27
• I think I have an issue here, I can't access to the transform between Frame 1 and end effector 1 at each time not as the manipulator 2. May 25 at 10:22