I would like to know how to go about evaluating 3D occupancy grid maps.
I have a SLAM system that produces a 3D OGM (in .bt format using octomap/octovis)
I also have a ground truth OGM in same .bt format.
How do I compare the accuracy of my map to the ground truth map in a qualitative and quantitative way?
Important notes:
- The two maps may not be the same scale.
- One map may be less dense than the other.
One method I have thought about using is MRPT's occupancy grid matching application This would require me to send both 3d maps as a message to the octomap_server node in ROS, get the resulting map in Rviz, save the image 2D image of each separately, and then somehow convert the images to MRPT's .simplemap file format, and then run MRPT's grid matching program on the two files.
Surely there is a better/more accurate way?
EDIT: So I did more research and another route I could go is Matthew's Correlation Coefficient (MCC). I could compare two maps and iterate over each cell to compare my result to a ground truth, counting the True and False Positives and Negatives. Only problem with this is that I have to assume that the two maps are the same scale, and also in the same orientation.
If you have any ideas on solving these scale and orientation issues don't be shy.