3
$\begingroup$

The dynamic programming algorithm refers to the Bellman equation. An open-loop control decides movement at the initial point while a closed-loop control decides control during the movement. Now most robotic application looks like closed-loop control: in every point, it checks how it is doing with respect to some reward function, this is my thinking. Now most participants in threads such as How mature is real-time programming in robotics? do not differentiate their scope, perhaps they haven't thought about it. Anyway, I am interested to know:

How is dynamic programming used in robotics? Is there any research about DP usage in robotics?

Improve this question
$\endgroup$
3
  • $\begingroup$ How is "dynamic programming" related to "real-time programming"? $\endgroup$
    – Shahbaz
    Apr 3, 2013 at 8:49
  • $\begingroup$ @Shahbaz "Dynamic programming" is a term invented by Bellman, a specific mathematical thing used in academia. The "real-time programming" may be or may not be related to it, I don't know. It looks like a term used by some people not understanding some applied mathematics. Can you explain me what you mean by real-time programming? It looks like closed-loop optimization problem. $\endgroup$
    – hhh
    Apr 3, 2013 at 12:30
  • 2
    $\begingroup$ Real-time computing is not a term used by people who don't understand applied mathematics. Real-time computing/programming is a part of computer science related to implementing and scheduling tasks that have a deadline. In particular, hard real-time tasks should not miss their deadlines under any circumstances. Real-time is often mistaken for "fast", which is an orthogonal concept. Closed-loop control is a very simple example of a real-time task. Dynamic program is a completely unrelated subject. $\endgroup$
    – Shahbaz
    Apr 3, 2013 at 13:18

1 Answer 1

Reset to default
1
$\begingroup$

I won't answer directly how Bellman equation is used in robotics but DP is used in many areas, see the below. Even though we have a dynamic system with velocity/acceleration, we can still use linear-approximation like linear-programming algorithms here. I provide below some material and lectures about Dynamic programming and robotics. This is pretty interdisciplinary area so hard to find concise description.

Areas

  • path-planning
  • vision such as stereo-vision
  • CSP and VCSP problems (see the cognitive robotics below)

Alert about terminology!

The term "dynamic programming" is a special term referring to special class of mathematical problems. The term "dynamic" system in physics refers to problems in continuum mechanics where objects are not in rest or in constant motion but they can accelerate. Then again Statics considers problems in stable equilibrium.

The term "dynamic programming" is programming for historical reasons but the basic idea is to break the large problem into small problems. It has analogues to fields such as Economics (for example sub-game-perfect equilibrium in game-theory).

Material

Path-planning

Vision, Stereo vision and Dynamic programming

Lectures

  • Underactuated robotics in MIT here -- this lecture here about DP

  • Cognitive robotics has some tasks about DP here

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ In the 1950s when dynamic programming came into existence, the term "programming" was often used synonymously with "optimization". This was the case with "dynamic programming". $\endgroup$
    – koverman47
    Jul 11, 2018 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.