Real-time object classification for an indoor autonomous quad-rotor

Question

I am designing an indoor autonomous drone. I am currently writing an object classification program in OpenCV for this purpose. My objects of interests for classification are: ceiling fans; AC units; wall and ceiling lamps, and; wall corners. I am using BoW clustering algorithm along with SVM classifier to achieve this (I'm still in the process of developing the code, and I might try other algorithms when testing).

The primary task of the drone is to successfully scan (what I mean by scanning is moving or hovering over the entire ceiling space) a ceiling space of a given closed region while successfully avoiding any obstacles (like ceiling fans, AC units, ceiling and wall lamps). The drone's navigation, or the scanning process over the ceiling space, should be in an organised pattern, preferably moving in tight zig-zag paths over the entire ceiling space.

Having said that, in order to achieve this goal, I'm trying to implement the following to achieve this:

1. On take off, fly around the given closed ceiling space and use SLAM to localise and map its environment.

2. While running SLAM, run the object classier algorithm to classify the objects of interests and track them in real time.

3. Once obtained a detail map of the environment and classified all objects of interest in the local environment, integrate both data together to form an unified map. Meaning on the SLAM output, label the classified objects obtained from the classifier algorithm. Now we a have fun comprehensive map of the environment with labeled objects of interest and real-time tracking of them (localization).

4. Now pick random corner on the map and plan a navigation pattern in order to scan the entire ceiling space.

So the question now here is, is using object classification in real-time will yield successful results in multiple environments (the quad should be able to achieve the above mentioned tasks in any given environment)?. I'm using a lot of train image sets to train my classifier and BoW dictionary but I still feel this won't be a robust method since in real-time it will be harder to isolate an object of interest. Or, in order to overcome this, should I use real-situation like train images (currently my train images only contain isolated objects of interests)?

Or in my is using computer vision is redundant? Is my goal completely achievable only using SLAM? If, yes, how can I classify the objects of interest (I don't want my drone to fly into a running ceiling fan mistaking it for a wall corner or edge). Furthermore, is there any kind of other methods or sensors, of any type, to detect objects in motion? (using optical flow computer vision method here is useless because it's not robust enough in real-time).

Any help and advice is much appreciated.

Answer 1

tl;dr - SLAM is good enough for your task.

The ceiling is the area where your robot will be traveling in. Few things to notice -

• Ceiling is fixed and not changing.
• Things on the ceiling are fixed as well at any given time unless the environment is changed.
• The number of things on the ceiling are also fixed.
• If you are going to be using a feature based map then the features are the walls and the objects on the ceiling all of them being obstacles.

In view of these three observations your SLAM should be enough, these obstacles are your features and thus there comes no question as to identifying them or classifying. Commonsense tells that any obstacle is a potential danger to the robot and thus is to avoided or maneuvered around. Depending on the size of the map and the number of obstacles the algorithms used for SLAM will have to be chosen obviously. If the task is to just navigate the ceiling around these features from point A to B then SLAM should be enough. As for detecting the fan, since you know the position of it on the ceiling, usually it is in the center, then u can feed that information into your map and let your robot maintain a certain radius away from that particular feature. This way you will avoid doing unnecessary computation in trying to keep track of frames or extra sensors to do this.

Object classification and tracking can be done since the scene your robot are static, but is not necessary for your task. In addition the training will have to be done for different environments which IMO doesn't seem to be very feasible and complicates things for your task.

Answer 2

If you choose to use real time object detection, consider the two following:

• Deep Drone: Object Detection and Tracking for Smart Drones on Embedded Systems

• YOLO: Real-Time Object Detection

The first is a project out of Stanford that is highly relevant to your project. They consider the use of a few different object detection strategies. The strategy I would recommend for your application is listed in the second bullet point. The YOLO object detector (now on version 3) is currently state of the art. Assuming you don't have powerful computing devices available to your UAV, you can use the YOLOv3-tiny. It is very quick, although at the cost of accuracy, but I expect it will be good enough for your application. You will be able to use one of these methods in real-time and in multiple environments if your training data set is robust enough.

Additionally, you can integrate a good object detection strategy within your SLAM technique. As a result, it would not be redundant.

On a side note, practical, real-time object detection wasn't truly first achieved until about two years ago (YOLOv1 and MultiBox SSD). As a result, there are new possibilities that haven't yet been fully explored, so I say have fun with it!