I am currently developing an off-road navigation system using NAV2 and various sensing modalities. Our setup includes two 3D Velodyne Lidars and a 2D SICK Lidar, each mounted at different orientations on our vehicle. Additionally, we have a FLIR monocular camera, a radar, and ultrasonic sensors.
Our Velodyne Lidars do not detect grass around the main field road as an obstacle, which causes path planning issues as the grass should ideally be treated as an obstacle. Therefore it would traverse the wrong path as display sketch below:
To overcome this, we're exploring the fusion of radar and camera data to obtain depth information. We have successfully segmented the image data obtained from the camera and would like to convert this segmentation into an occupancy grid map (either free or occupied space).
So, we have two distinct observation sources for generating the costmap:
Lidar data Fused camera-radar data (from segmented images) My questions to the community are:
- Is it feasible to create a costmap by combining these different data sources in ROS2?
- If so, should the data be fused before generating the costmap or can they be handled independently?
- What is the best approach to create a costmap based on camera data, radar data, and potentially other data types such as ultrasonic sensor data?
- Are there any existing ROS2 implementations or resources that discuss the creation of costmaps using diverse sensor data?
At the end we desire a costmap like image below could be generated, avoiding the planner to consider the grass as navigable free space.
Any insights or advice you could provide would be greatly appreciated. Thanks in advance for your time and help!