# How to estimate the position of multiple static ground targets captured from a down facing camera?

An aerial vehicle captures images of the ground using its down facing camera. From the images, multiple targets are converted from their pixel position to the camera reference frame using the pinhole camera model. Since the targets are static and there is information of the vehicle attitude and orientation, each sample is then converted to the world referencial frame. Note that all targets are on a flat, level plane.

The vehicle keeps "scanning" for the targets and converting them to the world referencial frame. Due to the quality of the camera and detection algorithm, as well as errors on the altitude information, the position of the "scanned" targets is not constant (not accurate). A good representation might be a gaussian distribution around the target true position, however it will also be influenced by the movement of the aerial vehicle.

What's the best approach to estimate the position of the targets from multiple readings? This basically resumes to a problem of noise removal (as well as outlier removal) and estimation, so I would like to know what algorithms and strategies could solve the problem. In the end I expect to implement and test a collection of different approachs to understand their performance on this specific problem.

Furthermore, this system is implemented using ROS, so if you know of packages that already do what I'm searching for I would be glad to hear. You can also cite papers on the topic that you think might be of my interest.

• I voted to close this because it's an unbounded design problem (you don't have a problem, you're just looking for advice). That said, the key phrase that I think will get you started is called structure from motion. Basically, the pixel location only corresponds to an angle based on the camera transform. You need to know angle and depth to get a real x/y coordinate. – Chuck Jan 22 '16 at 22:25
• The problem I need resolved is one of noise removal and estimation of multiple signals at the same time. I'm searching for help on resolving that issue, don't get why you call it an unbounded design problem. And what you proposed is actually the part that I already solved since I wrote that I already converted the position of the target in order to the world referencial frame. – nVolteX Jan 23 '16 at 12:23
• Can you please revise your question to include how you are currently getting measurements? The way that I had read your question, I thought you had an aerial vehicle with known X/Y/Z/$\theta$ and a camera, and with nothing else and only 1 frame you were trying to determine the X/Y/Z of ground targets, which isn't possible unless you have more information. What method are you using to determine ground coordinates? What kind of noise or measurement error are you seeing? – Chuck Jan 24 '16 at 0:10
• I added more info as you asked. – nVolteX Jan 24 '16 at 11:47

## 1 Answer

I see that you updated your question some, but maybe it's your scenario that's not clear. If your targets are all of about the same size, and all on a flat, level plane, then I can see that you could get their height by the the height of the aerial vehicle. I thought it was randomly distributed targets on an unknown surface, in which case the height of the aerial vehicle doesn't really buy you anything; you would need some other information to determine the 3D coordinates of the target. As it stands, knowing that the targets are all on a level plane is the extra information.

So, if you describe your noise as Gaussian, and the measurements are taken from a linear system (moving platform), then that's pretty much the definition of when to use a Kalman filter; "the [Kalman] filter yields the exact conditional probability estimate in the special case that all errors are Gaussian-distributed."

You'll need to know the covariance of the errors, but that shouldn't be too much of an issue as it sounds like you have already taken some measurements, enough to know that they are noisy and Gaussian in distribution.

• That's actually one of the approachs I had in mind, however with multiple targets I would need a way to associate each sample to the "correct target" (data association) without restrains on the distance between targets. Notice that the targets do not have unique IDs. – nVolteX Jan 25 '16 at 16:10
• @nVolteX - If the targets don't have unique IDs, then how are you comparing subsequent measurements between samples to determine that the position measurements are noisy? If your problem is needing to know how to associate measurements to a particular target, then you should ask it as a new question and link to this one to give context. – Chuck Jan 25 '16 at 16:49
• The problem is multi-target tracking/position estimation, which in order to be solved needs to go through some kind of "data association" and tracking/filtering. For example, if I could assume to have information about the number of targets then something like the k-means clustering would solve both problems. However the "hard" part, comes when I don't have prior information about the number of targets. – nVolteX Jan 25 '16 at 18:27