# Facial Detection combined with Path Planning using Kinect Sensor

I am planning to use a Kinect Sensor on a mobile robot for path planning based on facial detection.

Assuming I am able to detect the faces of 5 individuals in a room (both the environment and the people are static), can I somehow get the coordinates of said faces, and use those as reference points to draw a path the robot should follow (using a Nearest Neighbor algorithm for example) while stopping at those individual points for 5 seconds? Can you indicate me to any resources, papers, or GitHub repositories explaining it? Also, would I need any odometry or can it all be done through Computer Vision?

Note: once detected, the faces' coordinates need to be memorized as there is no guarantee that they will remain visible throughout the entire process.

If I understand your question completely it is easy! You are using Kinect, so you have direct distance between the Kinect mounted on your robot and the faces. If you get the robot starting point as (0,0,0) you can find the coordinates based on the distance and the rotation angle of the robot when it finds each faces.

Assuming I am able to detect the faces of 5 individuals in a room (both the environment and the people are static), can I somehow get the coordinates of said faces

Yes, since the Kinect has a depth sensor embedded. See https://pterneas.com/2016/08/11/measuring-distances-kinect/ as a vague intro. I would suggest you get the Kinect SDK working on your machine. There must be several resources online with tutorials on how to get this to work.

and use those as reference points to draw a path the robot should follow (using a Nearest Neighbor algorithm for example) while stopping at those individual points for 5 seconds?

Yes. But here you should decide on whether you are happy re-planning once you reach each of the points or you want to save the reference path, and resume with the next waypoint once you waited for 5 seconds. Moreover, since you have a non-flying robot, remember to project the xyz point into the 2D plane (i.e. just take xy coords). Also take into account that unless you use an optimization-based algorithm (accurately fed with your robot kinematic model) which can plan a drivable path throughout the set of points, you might end up with the wrong orientation at each stop.

Also, would I need any odometry or can it all be done through Computer Vision?

Having odometry data is essential in order to track where is your robot. Typically this is provided by sensors on the actuators. If you do not have such sensors, another option is to get "visual odometry" from the Kinect directly, but that is probably an advanced problem to tackle at your stage.

Note: once detected, the faces' coordinates need to be memorized as there is no guarantee that they will remain visible throughout the entire process.

That is something that affects your particular implementation. A simple C++ vector can do the job of saving the coordinates.