object level sensor fusion for multiobject tracking

Question

I want to fuse objects coming from several sensors, with different (sometimes overlapping!) fields of view. Having object lists, how can I determine whether some objects observed by different sensors are in fact the same object? Only then I can truly write an algorithm to predict future state of such an object.

From literature I read those 4 steps:

1. Plot to track association (first update tracks estimates and then associate by "acceptance gate" or by statistical approach PDAF or JPDAF)
2. Track smoothing (lots of algorithms for generating new improved estimate, e.g.: EKF, UKF, PF)
3. Track initiation (create new tracks from unassociated plots)
4. Track maintenance (delete a track if was not associated for last M turns. also: predict those tracks that were associated, their new location based on previous heading and speed)

So basically I am questioning point 1, acceptance gate. For a single sensor I can imagine it can be just a comparison of xy position of object and sensor measurement, velocity with heading eventually. My case is however, I have already ready object lists from each sensor in every cycle, there are some algorithms how to merge informations about an object collected by different sensors (great source is e.g. here: http://www.mathworks.de/matlabcentral/fileexchange/37807-measurement-fusion-state-vector-fusion), but question is how to decide which objects should be fused, and which left as they were? Fields of view may overlap partly, not totally.

Answer 1

This is called "data association" in tracking literature. When you measure the position of an object, you need to know which object it was you measured. If you can estimate this probability, then you are free to choose the most likely association. This is a heavily researched topic, but boils down to Bayesian analysis.

Here's a simple way:

Assume we have two objects $o_1$ and $o_2$, an estimate of their position $x_1$ and $x_2$, and a measurement $z$. Suppose $z$ is just a position measurement. We'd like to update $o_1$ or $o_2$'s estimated position, but we have no idea with object we just measured. So, we find the most likely object, and update that.

1. Estimate $p(z|x_1)$ and $p(z|x_2)$. The Kalman filter gives you the tools to do this.

2. IF $p(z|x_1)>p(z|x_2)$ AND $p(z|x_1)>\gamma$, then update $x_1$ using $z$, the Kalman filter gives you the tools to do this as well.

3. ELSE IF $p(z|x_2)>p(z|x_1)$ AND $p(z|x_2)>\gamma$, then update $x_2$ using $z$, the Kalman filter gives you the tools to do this as well.

4. ELSE, no probability is greater than $\gamma$, so we drop the measurement. You can set $\gamma$ to zero if you don't care.

Two things: First, you also need to know how to initialize an object. That's why $\gamma$ is important. If the measurement doesn't seem to match any known estimates, you might have just detected a new object.

Second, I have a suspicion that your question will be heavily edited, so I'm not putting a lot of details in just now.

---

EDIT To address your newer, revised question:

You have a different problem, but it is totally related. First, you want to fuse objects. This is as simple as estimating the probability that all measurements of object 1 and all measurements of object 2 are in fact of the same object. The algorithm is simple, but nearly intractable computationally. This is the same as track merger.

To really do this with objects in 3D, you need a model of the object which allows you to estimate the probability that two partially overlapping (or possibly just nearby) objects are in fact the same object). But this is exactly the same as the acceptance gate approach.

note

There are only a few really useful ways to mere measurements, and they are well defined. I strongly suggest if you wish to continue with abstract tracking and estimation tasks that you read Bar-Shalom's Tracking and Estimation book. Please get it from a library if you can. I just don't know of a better reference.

Answer 2

This depends on a lot of things. What kind of sensors? Are they stationary? Are they at least fixed with respect to each other?

For example, if you have a system of multiple cameras that are fixed with respect to each other, you can calibrate them, so that you know their relative positions and orientations. Then not only will you be able to determine whether two detections in different cameras are the same object, but you will be able to determine its position in 3D.

Here is the classic paper on camera calibration. To actually do the calibration you can either use OpenCV or the Caltech Camera Calibration Toolbox for Matlab. OpenCV has an example application for camera calibration which is easier to use than the Caltech toolbox.