0
$\begingroup$

Consider the dynamic system $\dot{x}=Ax+Bu$ and the optimal control formulation of $J = x^TQx + u^TRu$, with optimal linear feedback $u=Kx$ where $K$ is the gain matrix, and I use $k$ to denote time step from 0 to N.

From one book, I read the Riccati iteration derived from dynamic programming going backward from N to 0 (I use prime to denote matrix transpose below) as

$K(k)=-(B'\Pi(k+1)B+R)^{-1}B'\Pi(k+1)A$ with $k=N-1,N-2, ..., 0$ where $\Pi(k-1)=Q+A'\Pi(k)A-A'\Pi(k)B(B'\Pi(k)B+R)^{-1}B'\Pi(k)A$ with $k=N-1, N-2, ..., 0$ This is a time varying optimal gain, I guess $K(0)\neq K(k)$?

However, from another book, I read the algebraic Riccati equation using variational method, a matrix P has to be found satisfying $PA+AP^{'}-PBR^{-1}B^{'}P+Q=0$, then the optimal gain is $K=-R^{-1}B^{'}P$.

My question is, does this mean for discrete time, the optimal LQR gain should be time varying and for continuous time the optimal LQR gain should be a constant matrix? If so, what is the key insight causing such a difference ?

$\endgroup$
0
$\begingroup$

The algebraic Riccati equation is for the infinite horizon LQR. For finite horizon one can use the Riccati differential equation

$$ -\dot{P}(t) = P(t)\,A + A^\top P(t) - P(t)\,B\,R^{-1}B^\top P(t) + Q. \tag{1} $$

The reason why $(1)$ gives an expression for $-\dot{P}(t)$ is because one has to start the differential equation at the time horizon $T$ with $x(t)^\top P(T)\,x(T)$ the terminal state cost and $(1)$ is solved backwards in time until $t=0$.

Similarly, for discrete time one also has the infinite horizon LQR with an associated algebraic Riccati equation, and the finite horizon LQR with an associated Riccati difference equation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.