How much should I expect a Kalman filter to converge?

Question

I am learning about Kalman filters, and implementing the examples from the paper Kalman Filter Applications - Cornell University.

I have implemented example 2, which models a simple water tank, filling at a constant rate. We only measure the tank level, and the Kalman filter is supposed to infer the fill rate.

According to the model, the fill rate is a constant, so I assumed that over time, the Kalman filter would converge more and more accurately (and with less and less noise) on the correct fill rate. However, the amount of noise in the fill rate never seems to reduce after the first few iterations:

This graph shows how the fill rate part of the state vector changes over the course of 1000 iterations of the simulation.

Adjusting the Measurement Variance Matrix seems to have very little effect on the fill rate noise.

Also, the Kalman gain vector and State Variance matrix seem to be constant throughout the simulation. I assumed that the State Variance would reduce as the filter became more and more confident in its state estimate.

Questions: - Is this graph what I should expect to see? - Should the Kalman Gain vector and State Variance matrix change over time in this situation?

Answer 1

Let's assume a constant Kalman filter $x_n = Gx_{n-1}+\omega$ with $\omega \sim \mathrm{N}(0, W)$ and $y_n = Fx_n+ \nu$ with $ \nu \sim \mathrm{N}(0,V)$, and initial state $x_0 \sim \mathrm{N}(m_0, C_0)$, where $C_0$ is the initial state variance. Then, the sequence of state covariance matrices $C_0, C_1, \dots, C_n$ converges to a constant covariance matrix $C > 0$ when $n \to \infty$ if the system is observable, and $C$ is a function of the system and measurement covariance matrices $W$ and $V$, respectively (i.e., depends only on the signal-to-noise ratio). The Kalman gain also converges to a constant matrix. This convergence process occurs in just a few steps.

In summary, the state covariance matrix $C_n$ does not reduces indefinitely with time, it reduces until achieve a limiting constant covariance matrix $C>0$. In fact, it is not even sure that $C_n$ will decrease! If $C_0 < C$, the sequence $C_0, C_1, C_2 \dots$ increases until $C$.

See the second chapter of the following reference for a proof: Bayesian forecasting and dynamic models, by West and Harrison

Answer 2

Your graph is actually indicative of what I would expect to see, but there might be some problems. For now, let us assume that you've implemented it perfectly.

the fill rate is a constant, so I assumed that over time, the Kalman filter would converge more and more accurately

Does your Kalman filter know that the fill rate is constant? It's actually pretty easy to estimate fill rate given lots of data, but one that can estimate it as fast as possible is very powerful.

It looks like your graph starts at zero and then rings three times and converges very fast. At this point, it just responding to injected noise.

However, the trick is to balance favoring your current estimate vs incoming data using the Kalman gain. The noisier the data, the more you'll have to favor your estimate. However, favoring it more means you won't converge that fast (because you're essentially telling the filter that the fill rate doesn't change that fast.)

Two items to test:

1.) The filter is receiving a measurement z with added noise. Take that signal and plot it's derivative on top of the estimated value. Look at how much noise is in the signal.

2.) Halfway through your simulation, increase the fill rate by 2x and see how the filter responses. Then play with the values in K (move them away from their calculated optimal values) and see what the behavior is.

Answer 3

The reason the filter stops converging on a solution is because of the Process Variance Matrix, Q.

This matrix tells the Kalman filter how imperfect the model itself is. If the matrix contains only zeros, you're telling the filter that the model is perfect, but if it contains non-zeros, you're saying that the model can't entirely be trusted, and that more weight should be given to the measurements. This will then limit the filter's convergence.