# Why ODE for optimal control theory?

I am trying to understand optimal control theory which forms the base for reinforcement learning techniques in AI. Whenever I open a lecture or a book or any online notes, everything starts with an ODE and then derivation goes the payoff function which is straight forward.

I am trying hard to comprehend why an ODE models any system? Many say it easy to begin with but why this model?

$$dx/dt = f(x(t))$$

I could not find the reason and decided to ask for help.

• I don't understand your question. You need ODEs to mathematically describe how the system behaves as time goes. What else need to be said?! May be if you narrow your question a bit, ppl may help. Feb 16, 2017 at 14:30
• ODE stands for Ordinary differential equation which describes analog systems like curves. For controlling a robot that is not powerfull enough. So hybrid systems are extended by with finite state machines. On the one hand, you have a normal state-machine which iterates through the walk cycles and on the other hand inside every state a solver calculates Ordinary differential equations. (1) Feb 23, 2017 at 21:08
• Reinforcement Learning definitely isn't based upon continuous time optimal control theory. Jul 25, 2020 at 23:44

The reason why ODE's are used is simply: physics. It would be great if any system could be modelled by a simple linear function like $x(t)=at$, but nature is not so simple, or linear. Even when you neglect nature, dynamical systems, like $\dot{x}(t)=f(x(t))$ still pop up everywhere, like CroCo said, it is the basis of the mathematical modelling of many systems.

I would suggest looking into differential equations first before starting with reinforcement learning.

• I looked at one f the differential equation class on edx. Cleared it up. Thanks. Feb 16, 2017 at 19:29
• @bicepjai: What exactly was cleared up by the edx class? Please share the details of what you learned from the class that clarified your question.
– Paul
Feb 16, 2017 at 21:35
• Parameters on modeling depends on the system we are attempting to model which is f(x) here. Rate of change in a system is a natural way of modeling which gives us the necessary system state as function of time, convinced myself after sifting through examples. I ended up signing up for the mitx diff eqn course. Feb 16, 2017 at 21:44