How does dense mapping/voxel occupancy grid map work with bundle adjustment?

Question

When running for example some version of Orb Slam with stereo vision you're using sparse features for tracking but you can also infer a full depth map. I am curious how you use that full depth map to build a dense 3d map? Do you just run bundle adjustment on the sparse feature set to get the optimized positions and then re-run your depth perception algorithm on the key frames to generate a bunch of point clouds/voxels or is something else done?

Answer 1

Most commonly, for the new relative poses of the nodes containing depth data after you optimize your pose-graph, you re-generate your dense representation using that stored sensor data. So each time you optimize the graph, you're essentially generating a new dense representation instance. The same is done in most 2D laser scanner graph SLAMs to create the occupancy grid. This is typically why we try to reduce the frequency of rastering the occupancy grid (or in your case, the voxel grid or similar) as much as reasonably possible. That generation process is not cheap if you're doing probabilistic modeling of free / occupied spaces.