The dynamic programming algorithm refers to the Bellman equation. An open-loop control decides movement at the initial point while a closed-loop control decides control during the movement. Now most robotic application looks like closed-loop control: in every point, it checks how it is doing with respect to some reward function, this is my thinking. Now most participants in threads such as How mature is real-time programming in robotics? do not differentiate their scope, perhaps they haven't thought about it. Anyway, I am interested to know:

How is dynamic programming used in robotics? Is there any research about DP usage in robotics?

  • $\begingroup$ How is "dynamic programming" related to "real-time programming"? $\endgroup$
    – Shahbaz
    Commented Apr 3, 2013 at 8:49
  • $\begingroup$ @Shahbaz "Dynamic programming" is a term invented by Bellman, a specific mathematical thing used in academia. The "real-time programming" may be or may not be related to it, I don't know. It looks like a term used by some people not understanding some applied mathematics. Can you explain me what you mean by real-time programming? It looks like closed-loop optimization problem. $\endgroup$
    – hhh
    Commented Apr 3, 2013 at 12:30
  • 2
    $\begingroup$ Real-time computing is not a term used by people who don't understand applied mathematics. Real-time computing/programming is a part of computer science related to implementing and scheduling tasks that have a deadline. In particular, hard real-time tasks should not miss their deadlines under any circumstances. Real-time is often mistaken for "fast", which is an orthogonal concept. Closed-loop control is a very simple example of a real-time task. Dynamic program is a completely unrelated subject. $\endgroup$
    – Shahbaz
    Commented Apr 3, 2013 at 13:18

1 Answer 1


I won't answer directly how Bellman equation is used in robotics but DP is used in many areas, see the below. Even though we have a dynamic system with velocity/acceleration, we can still use linear-approximation like linear-programming algorithms here. I provide below some material and lectures about Dynamic programming and robotics. This is pretty interdisciplinary area so hard to find concise description.


  • path-planning
  • vision such as stereo-vision
  • CSP and VCSP problems (see the cognitive robotics below)

Alert about terminology!

The term "dynamic programming" is a special term referring to special class of mathematical problems. The term "dynamic" system in physics refers to problems in continuum mechanics where objects are not in rest or in constant motion but they can accelerate. Then again Statics considers problems in stable equilibrium.

The term "dynamic programming" is programming for historical reasons but the basic idea is to break the large problem into small problems. It has analogues to fields such as Economics (for example sub-game-perfect equilibrium in game-theory).



Vision, Stereo vision and Dynamic programming


  • Underactuated robotics in MIT here -- this lecture here about DP

  • Cognitive robotics has some tasks about DP here

  • 1
    $\begingroup$ In the 1950s when dynamic programming came into existence, the term "programming" was often used synonymously with "optimization". This was the case with "dynamic programming". $\endgroup$
    – koverman47
    Commented Jul 11, 2018 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.