What frame of reference is used during Visual Servoing?

Question

I'm new to the whole visual servoing area.

I'm now reading the tutorial Visual Servo Control Part I: Basic Approaches" and I don't understand something fundamental - what information is available to the robot?

• Is the 3D location of the tracked features in the current frame known?
• Is it known for the desired frame?
• Is it known for both?

If it's known for both - then what would be the best thing to do? Would it be to compute the current and desired 3D location and orientation of the robot, and plan an optimal path accordingly, essentially knowing everything in advance?

Also, in what sense could a control law (i.e translation + rotation path) be optimal for a visual servo?

Answer 1

Generally, visual servoing is a way to measure your position relative to features that are seen on the camera, without having to know your absolute position. In other words, registering those features with those in a global frame would be a separate process.

Remember that visual servoing is not limited to simply providing more data about your static environment -- it's entirely appropriate to use visual servoing to maintain a position relative to something else in the environment that might move.