# How exactly does sensor fusion work in Kalman filters?

I've been looking into implementations of Extended Kalman filters over the past few days and I'm struggling with the concept of "sensor fusion".

Take the fusion of a GPS/IMU combination for example, If I applied a kalman filter to both sensors, Which of these will I be doing?

• Convert both sensors to give similar measurements (eg. x, y, z), apply a kalman filter to both sensors and return an average of the estimates

• Convert both sensors to give similar measurements (eg. x, y, z), apply a kalman filter, and return an estimate of the sensor I trust more based on a certain parameter (measurement noise covariance)?

I realize this question already has an accepted answer, but I'd like to provide some additional input. The question of sensor fusion is a good one, but, depending on the application, you don't typically want to "convert" (i.e., integrate twice) the IMU to obtain xyz position. Frankly, in my experience, the best way to approach fusing GPS and IMU data for a mobile robot (quadcopter, unmanned ground vehicle, etc) is already provided within the EKF framework:

1. Identify a plant process model and associated process noise for the robot.

2a. Identify an observation model and associated measurement noise for the GPS

2b. Identify an observation model and associated measurement noise for the IMU

tl;dr: to answer your specific question: you'd be doing neither! The IMU provides acceleration corrections to the process model, and the GPS provides position corrections to the process model. The sensor fusion aspect is more or less a weighted average based on the process noise and measurement noise. Sure, a purely kinematic process model might act as a double integrator, but that's not the only choice of process models. It might sound like semantics, but I think it represents a fundamental misunderstanding of the Kalman filter to speak in terms of "converting" the IMU to provide xyz corrections.

The plant process model can be a simple kinematic model (constant velocity or constant acceleration) or a sophisticated dynamical model. It is generally best to start simple and add complexity later if required. Process model are typically developed from first principles (physics), but you can probably find some decent ones online if you get stuck. The same is true for observation models for a GPS and IMU sensors.

The measurement noise statistics can be found from the manufacturer data sheets or experimentally. [Aside: if you're working with real instruments, it's best to try to get your own data on the measurement noise. If a static test will give you representative data, set the sensor on the bench, collect a bunch of data, and determine the measurement noise statistics. Pro tip: the std dev of the noise (aka sigma) often varies inversely with the square root of the sampling frequency, e.g., sampling at 10Hz instead of 1KHz reduces sigma by an order of magnitude. You can see this mathematically by working out the expected values.]

With a GPS and IMU, a more sophisticated dynamical plant process model will likely just improve the estimate of linear velocity. Depending on the update rate of your GPS and how much you care about your velocity estimate, the added complexity of a dynamic model may be unnecessary for your given application. If you have a good linear-velocity sensor, a dynamic model is rarely required. Many papers have been written and accepted to respected journals and conferences with kinematic process models.

[Aside: The difficulty of dynamic models is that their accuracy depends on knowing: 1. the structure of the model. 2. the parameters (mass, drag, etc) used in the model. The structure of the model can often be developed from first principles, but parameter estimation can be difficult even for known model structures.]

The "black art" of the EKF is the choice of process noise. In the linear case, Kalman proved that the KF will converge regardless of the noise (we understand how to propogate Gaussian distributions through linear systems), but the same isn't true for the nonlinear extension. There are many papers written on the subject of estimating process noise for nonlinear models. This still an area of active research. My advice is just to play with the process noise until you achieve acceptable performance.

Also: the paper linked in the accepted answer is certainly interesting, but far from the simplest approach. Based on a quick scan, the purpose of that paper is to report a new method/approach for real-time rejection of bad data from the IMU or GPS based on fuzzy logic and "contextual variables." NB: Again, in the context of EKFs, "bad data" means data inconsistent with the expected measurement noise. This is an important problem to address because the accuracy (and even stability) of the EKF hinges on the idea that measurement data from the instruments have the noise characteristics that you specified.

This is a key important point because it means any additional instrument, no matter how bad the noise is, won't make the state estimate worse PROVIDED the measurement noise used in the EKF matches the noise from the actual measured data. NB. this can be proven mathematically for the KF, but not for the EKF, though it works as expected the vast majority of the time with reasonable process noise.

Instrument failure and outlier rejection are worth investigating and understanding...after you've already implemented an EKF that can handle asynchronous measurements and a bias in the IMU measurement. Start simple and add complexity.

Hope this helps.

As far as I'm aware, the second approach is generally preferred. The 'certain parameters' you mention are used to define a 'context' or 'validity domain' in which each one sensor or the other should be trusted more. These domains are often inherently fuzzy, due to the characteristics and limitations of the sensors involved.

An article titled 'GPS/IMU data fusion using multisensor Kalman filtering: introduction of contextual aspects', published in the June 2006 issue of Science Direct, describes the development of a multi-sensor fusion algorithm using GPS and an IMU exactly like the one you describe in the question.

• -1. OP's second approach literally trows away one measurement based on which sensor they consider more accurate, therefore it is by definition NOT sensor fusion.
– Chi
Jun 26 '19 at 3:42