Sending Universal Robots UR5e to specific place Using Camera

Question

I have a UR5e installed with a simple camera on its end effector. The robot is at a known position (Cartesian pose/angles and joint angles)

Using the camera I took a picture of a QR code and got the object's real life coordinates to the camera using OpenCV's SolvePNP. This gives me a 3x1 translation vector and a 3x1 rotation vector.

I'd like to tell the robot to move to that position in translation and angles.

I guess first step would be to inverse that result so I have the QR code's position from the camera's axes point of view.

How do I "add" this movement to the robot's arm?

(Using Python and ut_rtde)

Thank you

Answer 1

Just run some inverse kinematics and get the 6 joint angles that move the camera to the wanted position.

And find the example Python code that controls each motor angle of the robot and give your 6 values above.

Answer 2

If extrinsic parameters of camera (Transformation between end-effector and camera) are known, you know the transformation matrix between camera and robot base. So, definitely you know the T matrix between base and QR code in cartesian space, so you coulduse a function to move robot respect base using cartesian movement or by joints (calculating IK knowing T matrix between base and QR)