# Calculation of error, based on camera images

I am trying to measure the precision error of a gripper that is mounted to the flange of a 6 DOF industrial robot. The gripper-design is rather complex, therefore we would like to make sure it is operating precisely.

I think a simple way to measure the gripper-precision is to use a camera and visual markers on a piece that is gripped. See the following illustration of the setup. The colorful box mocks a part that is being gripped by the gray gripper that is mounted to the robot flange.

To get data to work with, I imagine the following procedure:

1. the robot picks up part 1
2. the robot holds the red side of the part in front of the camera, which takes a picture
3. the robot rotates the part by actuating its 6th joint by 90 degrees, the camera takes a picture of the green side
4. the robot rotates the part by another 90 degrees, the camera takes a picture (back side)
5. the robot rotates the part by another 90 degrees, the camera takes a picture
6. the robot places the part back in the nest
7. go back to step 1, until 10 iterations are completed

In summary: Each time the robot gripped the part, a batch of pictures from different sides are taken.

The error, that needs to be calculated, is defined as the pose-difference of the part that is caused by small errors in the gripper mechanism. The error has 6 dimensions.

The camera can be assumed to be calibrated.

Using two batches of pictures from two different grips: How can I use the image plane coordinates of the $$N$$ features attached to a part, to calculate the pose-difference of the part?

• Are you looking for absolute positioning in coordinate space, or repeatability? they are different for all robots generally you won't get much better than 0.1mm in absolute coordinates, repeatability of course can be around 10um...Otherwise, a 2d correlation between the photos and a comparison between the joints positioning (encoder positioning) would be my first thoughts....second..is it not possible to use an indicator? Nov 16 '19 at 0:34
• I want to measure and quantify the repeatability of the gripper attached to the robot. The gripper acts on parts that need to be positioned with high precision
– Alex
Nov 16 '19 at 17:27
• Positioning with robots is difficult beyond 0.1mm if you’re giving/expecting it to go to a prior generated coordinate...is that your case, or is the robots position already calibrated to begin with..or is that the assignment? Nov 16 '19 at 17:33

You could use the batch images from each color to correlate between them (only the mask of the correspondient color).

The correlation should be 1 if all the images are exactly the same, and less than 1 if they are different. so you could associate this "cost function" to map the result coefficient to the error of the repetibility.

Also you could apply correlation between landmark of the images, like borders, how much change this border position over the baches.

But the error will have the units on this new "cost function" you can't map it to milimiters without a mapping reference. Maybe you could associate each pixel with a length, but the preferible will be take at least one better resolution measurement to "calibrate" in a milimiters units.

I think that we could see each pixel as a frequency sample, so, by nyquist you will required at least the double of pixel by milimeter resolution.

For example, if you wish resolution of 7milimiters on the repeatability, then each pixel should be capable of capture less than 3.5mm

Rather than rotating the box, I would attach a calibration chessboard on the box to calculate the extrinsic. If you know the homography matrix between your camera and the board you can calculate the extrinsic(have a look at zhang camera calibration method).

Or using multiple cameras is one good way around. Place two cameras so that they can see each side of the box and calibrate their extrinsic within the range your robot holds the box.

I guess rotating 6th axis might not be a good idea as you need a calibration between the robot and camera.