In my application, my robot has the following physical setup:

  • Differential drive mechanics with feedback (wheel encoders)
  • Commercially available webcam mounted with a known transform to the base of the robot (RGB, no depth)

The robot will be navigating through a structured, indoor type environment (think office, home, or university), and I would like to be able to determine the navigable paths through the environment using my vision sensor.

What is the best way to approach the problem of finding safe paths to travel when given a single vision sensor?

Edit: I think that I am more interested in the vision processing techniques than the actual path-planning mechanics.

  • $\begingroup$ Are you allowed to change the environment to make things easier for the robot? For example, could you paint all of the baseboard trim in a high contrast color that is absent from the rest of the robot camera field of view? Could you paint or sticker the floor with a distinctive grid, or use alternating black and white floor tiles? $\endgroup$
    – Steve
    Commented Oct 2, 2017 at 18:20

2 Answers 2


Have a look at the literature that is available on this subject. In principle you can go two different ways: behavior based or sense/plan/act.

  • for the behavior based approach there are a lot of ways you could achieve your goal, and it also depends on your environment. One very simple and elegant solution that I saw (don't have reference at hand) was to assume pixels that are just in front of you to be ground, and extend the pattern through some sort of flooding algorithm. The projection of this onto the ground plane is traversable. Optical flow could be another possible solution, which takes a little more processing power, but might be more robust on textured environments. Have a look at the literature around, I am sure there a plenty more ways.

  • more complicated, but more robust would be some sort of structure from motion to get a model of your environment. This is not so trivial with a monocular camera, but possible. You could then generate a traversability map and perform path planning (some sort of A* or D*) and then path following on it.


You could have a look at visual servoing techniques with occlusion. Namely: try to reach "waypoints" on the floor or walls (e.g. doors, ...) using visual servoing.

  • 2
    $\begingroup$ This is a bit short for an answer. Could you fill in some more detail about visual servoing? $\endgroup$ Commented Oct 30, 2012 at 9:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.