Track a moving object

Question

If this has already been answered, by all means please point me to it.

I am in the process of building a quadcopter which I eventually plan to run autonomously by allowing it to track an object and take a video feed of it moving.

GPS is one of the options I've considered, basically:

• GPS antena on moving object (person, car, bike, surfer)
• GPS antena on quadcopter
• Radio to transmit coordinates from moving object to quad copter

Some of the challenges I can foresee are

• Line of sight for camera. How does the camera know exactly where to point?
• Angle, how can I pre-program the quad to always record, say... 10m to the right of the moving object, or even better, program a set of angles to record from whilst keeping up with the object
• GPS accuracy, what happens if the GPS lock is weak?

What are some of my other options? I saw this TED Talk where the quads are following a ball shaped sensor? I believe it uses Kinect cameras and lots of them which is not really an option for this challenge.

So I'm open to hearing some ideas before I start research and development of these features.

Answer 1

A Vicon motion capture system system is used in the TED Talk that you referenced. It is similar to a Kinect in that gives the 3D coordinates of any object being tracked within its field of view. Like the Kinect it uses IR, but uses IR reflective balls attached to the quadrotor and the presenters pointer to identify and track objects. Not to mention it is a considerably more expensive system.

Systems like Vicon and the Kinect do not work outdoors because they use IR and the sun blinds them. The GPS solution you propose is a common approach to this problem for outdoor venues. Another approach might be to use Quuppa, an RF based motion capture system.

Regardless of which tracking method you use, once you know the position of the robot and the target the problem of calculating the direction to the target, or ahead of it, is a basic geometric problem. Subtracting the robot position from the target position will give you the direction to point the camera. To track a few feet ahead of the target you would first add vector to the target position representing the direction of travel and lead distance (10m in your example).

If you don't actually need to know the location of the robot, only its relative position to the target, then you could use computer vision techniques to track the target and move with it. Of course you will need to avoid obstacles in the process. Also the obstacles themselves could occlude your view in which case you would need some way to predict the targets movement. This could be particularly difficult to come up with if your tracking humans outdoors.

The reality is that the problem you are addressing is a highly active area of research. I know of several research groups, both in academia and industry, that are trying to address this problem. I have only scratched the surface here.

Answer 2

Research the accuracy of your proposed GPS receivers. You may be disappointed. You can augment GPS accuracy by fusing the GPS receiver output with data from an inertial sensor, but as far as I know you can't get a hobbyist-price system where that's already done, and the problem isn't at all trivial to solve.

For pointing, consider a compass on your quad copter.

It's going to be a pretty rough problem to solve, particularly if you want the quad copter to be small and not tied to a ground station (there's lots you could do if you had video processing capabilities, and the time to get them working, but that's not something that'll readily fit on a small copter).

Answer 3

In this case, your absolute-positioning technology is a red herring. If you are trying to track an object, all that matters is your relative position. We did such vision-based navigation (relative to an underwater structure) with an underwater vehicle -- unable to receive GPS signals while submerged.

Assuming that you've established the ability to locate an object in a video frame (and deduce your range and bearing relative to it), all that remains is to calculate your desired position and feed that into your X/Y PID control. The only difficult part is getting the object into the frame initially. (Presumably, your coarse GPS control will get you in the right ballpark.)

I can think of 3 fairly simple ways that you might get your range and bearing from an input video stream. The best way would be to have native 3D support in the camera (such as in a Kinect). Failing that, if you know your target's size (and how to detect it in the frame) then you should be able to calculate the range based on how many pixels it occupies. Worst case, just use an augmented reality glyph on your target -- they are fairly easy for computer vision to detect.

Answer 4

Your proposed GPS solution is the best option by a significant margin, as long as you are in an environment where GPS is available (aka you are always outdoors).

Vision-based solutions might seem tempting, but they are really not appropriate in this situation. Not only are you limited in terms of camera quality and processing power, but object tracking in arbitrary locations outdoors is a horrendously difficult problem, especially once you start have partial/complete occlusions.

You also have a very nice advantage because the vast majority of GPS errors will affect both receivers simultaneously. Their "global" position might be off by dozens of meters, but their relative locations are likely to be much, much closer. (This is essentially DGPS.)

As soon as you know the location of your quadrotor relative to the location of the target the problem reduces down to high-school trig or basic linear algebra.

Note that as soon as you start talking about obstacle avoidance, or robustness when sensors drop out things become genuinely complicated again.

Frankly, a reliable solution to this problem that does not involve placing a beacon on the target would be at the minimum worthy of a PhD thesis, and probably worth a lot more.

Answer 5

If you want sub-meter tracking of the moving object, GPS alone will not be sufficient. I concur that passive vision-based techniques will require a huge amount of engineering to implement, not to mention substantial size, weight, and power. I recommend using differential GPS techniques -- not exactly easy, but simpler from a system standpoint. You will need a high-update rate GPS receiver that outputs Carrier Phase and Pseudo-Range data as well as a traditional GPS solution. You will also need to transmit this additional data to your quadcopter from both a nearby reference station and the moving object. It is useful to have a stationary reference station since its position is exactly known, so using the satellite ephemeris you can determine the propagation and timing errors from each satellite very precisely. Since you know the approximate geometry to both the non-moving reference station and the moving object, and the paths to the satellites are approximately the same for both cases, you can adopt the precisely calculated propagation and timing errors learned from the reference station and apply them to the moving object. Additional accuracy is gained by computing a relative position solution for both the quadcopter and the moving object that explains the observed changes in Carrier Phase over time as both of these move. Since these are phase changes, the integer number of cycles is unknown and must be solved for using a well-known technique for finding integer solutions to least-square problems (name escapes me at the moment). A Kalman Filter or Particle Filter is also typically needed to converge to the correct position solution using the carrier data after fusing with the pseudo-range-derived data. Check out work done at Auburn University's Navlab for examples of this type of work.