# Kalman filter for visual tracking of a ball sliding on a gutter

I'm working on a project where a robot needs to keep a ball at a desired position on a gutter. The gutter is fixed at one end and held at the other end by the robot’s effector which therefore decides of its orientation and, as a consequence, of the position of the ball. The robot's effector can move up or down vertically. The robot's controller is deep learning based and was trained in simulation.

Once trained, I deployed it on the physical robot to perform the task on the real world. The position of the ball and velocity of the ball are both needed by the controller. The ball position is obtained through visual tracking with a low quality webcam which means that the observations are noisy. The velocity is obtained by computing the difference in ball position between two timesteps (~0.1s). However, noise in the ball position produce large error in the velocity computation. Even if the controller is performing ok with noisy observations, I'd like to improve it. So I started looking into the Kalman filter.

My state is composed of the ball position (1d) and velocity (1d), however I'm measuring the ball position only with the camera. My state model looks like this :

$$\binom{x_k}{v_k} =A\binom{x_{k-1}}{v_{k-1}}+B.U$$

with

$$\begin{cases} A = \begin{bmatrix} 1&dt\\ 0&1\end{bmatrix} \\ B=\binom{0.5dt^2}{dt} \\ U=a \\ dt=0.1 \\ \end{cases}$$

For the command $$U$$, I considered 4 cases (depending on of the direction of the ball and the orientation of the gutter) and derived the acceleration for each : $$u$$ is the effector height (which is controlled) below or above the equilibrium position of the gutter and $$\mu = 0.06$$ is the friction coefficient plastic/plastic (both ball and gutter are in plastic) that I found to be given the most realistic behavior (see figure below). In code I'm checking, the sign of $$v$$ and $$\theta$$ to know which equation to use. I recorded a trajectory of the robot performing the task to validate the model (prediction step). However, the model doesn't seems to be very accurate : The predicted ball position doesn't fit the true (measured) position at all. However the predicted speed is smoother than the naively computed one (measured). So my first question is : Is something wrong with my prediction step ? Even when adding air friction the predicted ball trajectory is still very wrong. Is the Kalman filter still able to work with such an inaccurate model ?

• For the signals filtered by the Kalman filter did you use both the prediction and correction step? And what covariance matrices did you use in the Kalman filter? Feb 2 at 19:39
• Thank you for your time. The curve are mislabeled on the figure. kf position and kf velocity should be labeled respectively predicted position and predicted velocity since they were obtained with prediction only. I'm new to Kalman filter so i'm still struggling to understand how to pick the values for the covariance matrices. For now I'm just trying to build an accurate prediction step. Feb 2 at 20:05
• A Kalman filter usually only tracks well when you include the correction step as well. Your model isn't perfect, but I still suspect you can get a much better velocity estimate from it. Feb 3 at 6:51