# Why ODE for optimal control theory

I am trying to understand optimal control theory which forms the base for reinforcement learning techniques in AI. Whenever I open a lecture or a book or any online notes, everything starts with an ode and then derivation goes the payoff function which is straight forward.

I am trying hard to comprehend why an ODE models any system ? Many say it easy to begin with but why this model ?

$dx/dt = f(x(t))$

I could not find the reason and decided to ask for help.

• I don't understand your question. You need ODEs to mathematically describe how the system behaves as time goes. What else need to be said?! May be if you narrow your question a bit, ppl may help. – CroCo Feb 16 '17 at 14:30
• ODE stands for Ordinary differential equation which describes analog systems like curves. For controlling a robot that is not powerfull enough. So hybrid systems are extended by with finite state machines. On the one hand, you have a normal state-machine which iterates through the walk cycles and on the other hand inside every state a solver calculates Ordinary differential equations. (1) – Manuel Rodriguez Feb 23 '17 at 21:08

The reason why ODE's are used is simply: physics. It would be great if any system could be modelled by a simple linear function like $x(t)=at$, but nature is not so simple, or linear. Even when you neglect nature, dynamical systems, like $\dot{x}(t)=f(x(t))$ still pop up everywhere, like CroCo said, it is the basis of the mathematical modelling of many systems.

I would suggest looking into differential equations first before starting with reinforcement learning.

• I looked at one f the differential equation class on edx. Cleared it up. Thanks. – bicepjai Feb 16 '17 at 19:29
• @bicepjai: What exactly was cleared up by the edx class? Please share the details of what you learned from the class that clarified your question. – Paul Feb 16 '17 at 21:35
• Parameters on modeling depends on the system we are attempting to model which is f(x) here. Rate of change in a system is a natural way of modeling which gives us the necessary system state as function of time, convinced myself after sifting through examples. I ended up signing up for the mitx diff eqn course. – bicepjai Feb 16 '17 at 21:44