Visual servoing - tracking a point

Question

I am trying resolve some issues i am having with some inverse kinematics.

the robot arm i am using has a camera at the end of it, at which an object is being tracked. I can from the the camera frame retrieve a position, relative to that that frame but how do i convert that position in that frame, to an robot state, that set all the joint in a manner that the camera keep the object at the center of the frame?...

-- My approach --

From my image analysis i retrieve a position of where the object i am tracking is positioned => (x,y) - coordinate.

I know at all the time the position (a) of the end tool by the T_base^tool - matrix, and from the image analysis i know the position (b) of the object relative to the camera frame for which i compute the difference as such c = b - a.

I then compute the image jacobian, given the C, the distance to the object and the focal length of the camera.

So... thats where i am at the moment.. I am not sure whether the position change retrieved from the cam frame will be seen as position of the tool point, at which the equation will become un undetermined as the length of the state vector would become 7 instead of 6.

The equation that i have must be $$J_{image}(q)dq = dp$$

• J_image(q)[2x6]: being the image jacobian of the robot at current state q
• dq[6x1]: wanted change in q-state
• dp[2x1]: computed positional change...

Solution would be found using linear least square.. but what i don't get is why the robot itself is not appearing the equation, which let me doubt my approach..

Answer 1

I think you took a wrong turn in your approach.

I know at all the time the position (a) of the end tool by the T_base^tool - matrix, and from the image analysis i know the position (b) of the object relative to the camera frame for which i compute the difference as such c = b - a.

You don't care about position (a). You care about the position in the center of the camera frame, which you can think of as an imaginary end tool. Let's call it position (i). Just like your real end tool, this point will always occupy the same spot in the camera frame (which happens to be the center, in this case).

So, given position (b) and position (i), you can compute the distance between the two points. Your state vector will be 6 points -- replace the data for the "real" tool with the data for the "imaginary" tool in your inverse kinematics equation.

Answer 2

For a 6 degree of freedom manipulator there are an infinite number of (positions + orientations) which will keep a point object centered in the frame.

When you determine the position of the object relative to the camera frame, is it allowable to ignore orientation changes and move the camera (keeping its current orientation) to track the object? Or alternately, could you keep the origin of the camera frame constant and use rotations only to track the object? You could do either of these given the object location relative to the camera frame through the inverse kinematics computations.

You may consider including in your image analysis algorithm the computation of the object's position AND orientation relative to the camera frame. Then you will have a countable number of possible inverse kinematic solutions.

Once you determine which method (above) you are going to use, the inverse kinematics can be position-based, or velocity-based. For small relative motions like I think you're describing, I would use velocity control to keep the object centered. This involves computing the Jacobian matrix, and using it to set joint speeds to move the camera in the desired direction. The concept is simple: it is a mapping of $$\dot{ \vec x} = J \dot{ \vec q}$$ where $\vec q$ is your vector of joint angles, and $\vec x$ is your vector of Cartesian positions and orientations. However, the terms in $J$ can be rather complicated.

So your solution, using velocity control, requires you to determine the desired change in camera position/orientation, which is then applied to the equation above to compute the required changes in joint position.