6
$\begingroup$

When you've created a map with a SLAM implementation and you have some groundtruth data, what is the best way to determine the accuracy of that map?

My first thought is to use the Euclidean distance between the map and groundtruth. Is there some other measure that would be better? I'm wondering if it's also possible to take into account the covariance of the map estimate in this comparison.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ This is an interesting problem. Be aware that you might fall into a fractal problem where having more points gives you (falsely) a larger error. In other words, your increased precision works against you. $\endgroup$
    – Ian
    Dec 28, 2012 at 18:29

1 Answer 1

Reset to default
5
$\begingroup$

Assuming the map is a point cloud and that you know the alignment between the ground truth data and the map then calculating the mean squared error (MSE) would give you a relative understanding of the accuracy. A lower MSE would indicate they are very similar, 0 of course mean identical, and a high MSE would idicate they are very different.

If you do not know the alignment between the ground truth and the map then you could use expectation maximization over the alignments to find the best fit and then use MSE.

Improve this answer
$\endgroup$
2
  • $\begingroup$ My map is a vector of points in 2D, but I assume it's the same in the 3d case: <f1x, f1y, (f1z),...fnx, fny, (fnz)>. Is this the same as a point cloud? The wikipedia article suggests it is, but this is the first time I've come across the term. Also, by alignment, do you mean the correspondence between map features and groundtruth landmarks or something else? $\endgroup$
    – munk
    Dec 28, 2012 at 18:00
  • 1
    $\begingroup$ Yes you effectively have a point cloud where all the z components equal 0. By alignment I mean the relative placement and orientation of the two. In effect this tells you the correspondence between map features and ground truth landmarks. However, you can work backwards. I.e. if you know the correspondence between the map features and the ground truth features then you can extract the alignment. The problem in doing so is the possible error in your data. Expectation maximization (EM) would let you find the alignment that minimizes the difference between the two. $\endgroup$
    – DaemonMaker
    Dec 28, 2012 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.