Kalman filter with impulse noised outliers : A robust sequential algorithm to filter data with a large number of outliers

TL;DR

This paper introduces a robust Kalman filter variant designed to effectively handle data contaminated with a high proportion of impulsive outliers, outperforming classical methods in robustness and efficiency.

Contribution

A new model for impulsive noise outliers based on latent Gaussian processes and a fast filtering algorithm with outlier detection capabilities.

Findings

Outperforms classical methods in robustness and efficiency

Effectively detects and filters high outlier proportions

Successfully applied to real-world weighing system data

Abstract

Impulsed noise outliers are data points that differs significantly from other observations.They are generally removed from the data set through local regression or Kalman filter algorithm.However, these methods, or their generalizations, are not well suited when the number of outliers is ofthe same order as the number of low-noise data. In this article, we propose a new model for impulsenoised outliers based on simple latent linear Gaussian processes as in the Kalman Filter. We present a fastforward-backward algorithm to filter and smooth sequential data and which also detect these outliers.We compare the robustness and efficiency of this algorithm with classical methods. Finally, we applythis method on a real data set from a Walk Over Weighing system admitting around 60% of outliers. Forthis application, we further develop an (explicit) EM algorithm to calibrate some algorithm parameters.