# How to determine quality of ICP matches?

In SLAM frontends which use the Iterative Closest Point (ICP) algorithm for identifying the association between two matching point clouds, how can you determine if the algorithm is stuck in a local minimum and returns a wrong result?

The problem is defined as matching two pointclouds which are both samples of some arbitrary surface structure, and the sampled areas have an overlap of 0-100% which is unknown. I know the Trimmed ICP variant works by iteratively trying to determine the overlap, but even this one can be stuck in a local minimum.

A naive approach would be to look a the mean square error of the identified point pairs. But without some estimate of the sampling this seems a risky thresholding. In the manual for the Leica Cyclone they suggest manual inspection of the pair error histogram. If it has a Gaussian shape the fit is good. If there is a linear fall-off the match is probably bad. This seems plausible for me, but I've never seen it used in an algorithm.

• Jakob, did you ever get to the end of this? Facing similar problem, would love to hear what you've learnt in the process. Oct 17, 2016 at 19:31
• No, as far as I am concerned this is still open. Oct 18, 2016 at 6:31

Most of the straightforward ICP methods use a Least-Squares type approach. It is common and easiest to model when assuming a Gaussian error model corrupts the point cloud data. In this case the least squared fitting component of the ICP algorithm produces a Gaussian error model for its solution parameters with estimated variance.

That is, if you have access to the error's after matching, then you can estimate a Gaussian error on your transform's parameters in the same way you'd estimate the error in any other linear regression.

• Using a threshold on the least square error was what I was referring to in the question. I used it as well in applications, but it seemed like a very brittle parameter which is quite scenario/environment specific. Nov 8, 2012 at 11:39

In case you have some information from other sensors available (e.g. odometry from wheel encoders) you could use this when the rigid body transformation that the laserScanner suggests is far off.

Remember that on the long trajectories the odometry path diverges from the ground truth but locally it is pretty accurate.

PS. This is quite an interesting question so do let us know how you did it in case you actually solved the problem..

I think the best approach would be to use a data set which contains a ground truth. The data set which is most often cited in the literature is described in the paper "A benchmark for the evaluation of RGB-D SLAM systems." They also describe a few metrics to compare your pose estimation result with the ground truth. Hope this helps. Happy coding.

• Hey, thanks for the answer, though not quite what I was looking for. I am interested in knowing the quality of the match when there is no ground truth available. This is relevant for rejecting ICP results. Nov 7, 2012 at 21:17