In SLAM frontends which use the Iterative Closest Point (ICP) algorithm for identifying the association between two matching point clouds, how can you determine if the algorithm is stuck in a local minimum and returns a wrong result?
The problem is defined as matching two pointclouds which are both samples of some arbitrary surface structure, and the sampled areas have an overlap of 0-100% which is unknown. I know the Trimmed ICP variant works by iteratively trying to determine the overlap, but even this one can be stuck in a local minimum.
A naive approach would be to look a the mean square error of the identified point pairs. But without some estimate of the sampling this seems a risky thresholding. In the manual for the Leica Cyclone they suggest manual inspection of the pair error histogram. If it has a Gaussian shape the fit is good. If there is a linear fall-off the match is probably bad. This seems plausible for me, but I've never seen it used in an algorithm.