Move gantry system to detected position in image plane

Question

I have the following simplified gantry system (cartesian robot) that can move in the x axis direction:

My goal is to position the red pointer on top of the target (i.e., I want the real world x-coordinates of the pointer and the target to be the same). I have a vision system above the gantry and I am using two aruco markers. This way I can detect the position of the target and the carriage in the image plane. Hence, I know their x-coordinates in pixels.

Knowing the positions of the target and carriage arucos, how can I compute the position in the image plane (x-coordinate in pixels) where the carriage aruco should be after moving the gantry and positioning the red pointer just on top of the target aruco?

Any advice on how to make the system more accurate/robust?

Answer 1

You could do a manual calibration:

• Measure a far left x_target aruco in camera pixels (x_pixel_left),
• Put the pointer on that target and measure the corresponding carriage aruco position in camera pixels (x_carriage_left),
• Do the same for a far right position (x_pixel_right and x_carriage_right),

And then at runtime:

• Find the actual x_target in the camera image,
• Calculate the ratio of this x_target pixel position wrt x_pixel_left and x_pixel_right,
• Calculate the target carriage pixel position using the same ratio wrt x_carriage_left and x_carriage_right,
• Move your carriage accordingly.

If this is not accurate enough, you could add extra calibration points between far left and far right.

Alternatively, you could do a full camera calibration but that is more involved, and probably overkill unless your goal is to extend this to multiple degrees of freedom.