1
$\begingroup$

I am working on a "Dynamic Object detection" problem using just point cloud and deep learning algorithms. I have to use a static depth camera/ static lidar in the environment. The environment includes a certain number of dynamic objects like a Human, a car, etc., and also static objects, and the lidar is fixed at a point in the environment. How can I create a point cloud for such an environment? What should be the general approach for detecting the object?

If anyone can suggest to me a good starting point for working with lidar in gazebo and ros2, that would be also highly appreciated. Thank you!

$\endgroup$
2
  • $\begingroup$ Can you please elaborate more on what task is this actually for? Is that indoor/outdoor environment? What are all the classes you need to be able to detect in the point clouds? Is vehicles, pedestrians and cyclists enough? What lidar/depth camera are you using? Is it important to for the detection algorithms to run in real-time? What is your processing hardware? $\endgroup$ Sep 30 at 16:35
  • 2
    $\begingroup$ You shoiuld really only ask one questions at a time. There is little cross over between 'how to create a point cloud' and 'how to do object detection from a point cloud' so answers will be all over the place...that said I will suggest an answer for object detection. $\endgroup$
    – billy
    Sep 30 at 19:08

1 Answer 1

2
$\begingroup$

For object detection in a point cloud, I'll propose a process for you to follow instead of just providing links. I assume you don't have experience with machine learning and that is driving your question. If I'm wrong about that, my apologies.

I would turn the 3D point cloud into a depth map oriented with the viewer at the lidar. This would give you a 2D 'image' with each pixel representing depth. Use an available ML object detection algo like 2D YOLO that is pretrained and well documented. Study the tutorials, get it working, and then train the last few layers on training data you would label by hand.

https://learnopencv.com/train-yolov8-on-custom-dataset/

The results from 2D processing like YOLO may or may not be good enough for your case but at least you have learned a bit about machine learning, have your PC setup for machine learning, and understand labeling data sets.

Having experience with the easy stuff above, what you find with google will be easier to understand: https://cs230.stanford.edu/projects_fall_2019/reports/26232893.pdf

https://www.mathworks.com/help/lidar/ug/object-detection-with-point-clouds.html

https://medium.com/@regis.loeb/playing-with-point-clouds-for-3d-object-detection-eff1d98e526a

https://towardsdatascience.com/lidar-point-cloud-based-3d-object-detection-implementation-with-colab-part-1-of-2-e3999ea8fdd4

https://www.mdpi.com/2076-3417/13/11/6754

All of this is to explain that you're not going to get a suitable answer without some underlying understanding of the subject. If you want to use ML for object detection, learn about object detection and machine learning first, then try to leverage the work of those coming before you.

I have integrated YOLOV4 running on DARKNET into a ROS2 node for realtime 2D custom-object detection, but I started with learning the DARKNET separately and only after I was comfortable with it, did I think about integrating with ROS. It was more work than I expected (but I suck at software and it may be faster for you).

$\endgroup$
1
  • 2
    $\begingroup$ Really like this answer. Once OP reaches the point of considering making detections in 3D data, I'll throw in one more link: github.com/open-mmlab/OpenPCDet. Quite easy to make a wrapper ROS node around this and it kinda works out of the box. But this assumes the task is to detect mainly vehicles. $\endgroup$ Sep 30 at 21:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.