Tracking objects from camera; PID controlling; Parrot AR Drone 2

Question

I am working on a project where I should perform object tracking using the camera of Parrot AR Drone 2.0. So the main idea is, a drone should be able to identify a specified colour and then follow it by keeping some distance.

I am using the cvdrone API to establish communication with the drone. This API provides function:

ARDrone::move3D(double vx, double vy, double vz, double vr)

which moves the AR.Drone in 3D space and where

• vx: X velocity [m/s]
• vy: Y velocity [m/s]
• vz: Z velocity [m/s]
• vr: Rotational speed [rad/s]

I have written an application which does simple image processing on the images obtained from the camera of the drone using OpenCV and finds needed contours of the object to be tracked. See the example below:

Now the part I am struggling is finding the technique using which I should find the velocities to be sent to the move3D function. I have read that common way of doing controlling is by using PID controlling. However, I have read about that and could not get how it could be related to this problem.

To summarise, my question is how to move a robot towards an object detected in its camera? How to find coordinates of certain objects from the camera?

Answer 1

On the condition that you don't care about the position of the quadrotor regarding your target, but instead only of the absolute distance, I would say that for starters, you can extract the distance information from your OpenCV algorithm.

Have a target with a clear contour (eg a coloured ball) and find its relative position from the center of the camera frame, as well as the size of the contour. You don't need features to extract your relative orientation to the object, just know where you should turn to and how far it is. The size of the contour will need to be calibrated so that you can extract distance information from it (the farthest you are, the smaller the object). The position of the target from the center of the camera frame will tell you where you need to point your quadrotor.

After that, it's a matter of finding a control law for your quadrotor, which will minimize the offset of the target from the camera center, as well as keep it in the required size in the camera frame. This won't necessarily be easy, since there will be conflicting requirements. For example, you will want to bring the quadrotor closer to your target by pitching it down, but at the same time this will increase the targeting error. An MPC controller might be suitable for balancing those weights.

In all, I would say that, as it is presented, this is a full-fledged thesis project.

Answer 2

A standard approach (using opencv solvePnP) is using at least 4 points in the image that define landmarks of a known geometry. You can then get the pose of the camera relative to the object.

For example if you had a blue rectangle of which you could detect the corners in the image, and you knew the dimensions of this rectangle, you could work out the relative pose. You must have some way to identify which corner is which (say you had them each painted different colours just in the corners).

The relative position between your robot would be a (rotated) error, which you could feed into a PID.

See also: https://stackoverflow.com/questions/16265714/camera-pose-estimation-opencv-pnp