0
$\begingroup$

I am currently developing an off-road navigation system using NAV2 and various sensing modalities. Our setup includes two 3D Velodyne Lidars and a 2D SICK Lidar, each mounted at different orientations on our vehicle. Additionally, we have a FLIR monocular camera, a radar, and ultrasonic sensors.

Our Velodyne Lidars do not detect grass around the main field road as an obstacle, which causes path planning issues as the grass should ideally be treated as an obstacle. Therefore it would traverse the wrong path as display sketch below:

enter image description here

To overcome this, we're exploring the fusion of radar and camera data to obtain depth information. We have successfully segmented the image data obtained from the camera and would like to convert this segmentation into an occupancy grid map (either free or occupied space).

enter image description here

So, we have two distinct observation sources for generating the costmap:

Lidar data Fused camera-radar data (from segmented images) My questions to the community are:

  1. Is it feasible to create a costmap by combining these different data sources in ROS2?
  2. If so, should the data be fused before generating the costmap or can they be handled independently?
  3. What is the best approach to create a costmap based on camera data, radar data, and potentially other data types such as ultrasonic sensor data?
  4. Are there any existing ROS2 implementations or resources that discuss the creation of costmaps using diverse sensor data?

At the end we desire a costmap like image below could be generated, avoiding the planner to consider the grass as navigable free space.

enter image description here

Any insights or advice you could provide would be greatly appreciated. Thanks in advance for your time and help!

$\endgroup$
1
  • $\begingroup$ On Stack Exchange it is generally better to ask just one question per post, otherwise providing an answer that covers all points raised becomes rather difficult. $\endgroup$ Commented Jun 29, 2023 at 19:39

1 Answer 1

0
$\begingroup$

Is it feasible to create a costmap by combining these different data sources in ROS2?

Yes, that is common and pretty standard. The costmaps can take in N topics with N sets of data. That's kind of the point of the costmap - to take a bunch of data and create a globally consistent representation for use in planning and control.

If so, should the data be fused before generating the costmap or can they be handled independently?

Depends, but unless you have somethings you want to combine them beforehand to do, the costmap can buffer in the sensor data and combine them at layered costmap level.

What is the best approach to create a costmap based on camera data, radar data, and potentially other data types such as ultrasonic sensor data?

Use the appropriate costmap layers or create custom layers as needed. Sonars / lidars / depth cameras are supported out of the box.

$\endgroup$
3
  • $\begingroup$ Thanks for the feedback Steve! $\endgroup$ Commented Jun 29, 2023 at 22:26
  • $\begingroup$ Steve, have you are you aware some packages to convert segmented images to costmap data (free or occupied) ? , Even though using different sources to generate the costmap, all of them would consider the grass as free space to navigate... I see this pixel conversion as a solution. What are your thoughts? $\endgroup$ Commented Jun 29, 2023 at 23:13
  • $\begingroup$ Its been the topic of conversation for awhile and a number of people have done it, but I don't have a precise recommendation. There are several ways to convert the 2D mask to their 3D coordinates depending on the sensors you have on your robot, but I haven't had anyone tell me that 1 method was objectively better than the others. But, this would be a consequential contribution back to Nav2 if you made such a thing! $\endgroup$ Commented Jun 30, 2023 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.