A Framework for Robust Assimilation of Potentially Malign Third-Party Data, and its Statistical Meaning

TL;DR

This paper introduces a robust particle filter-based framework for sensor data fusion that incorporates statistical hypothesis testing to reject faulty data, enhancing real-time state estimation in uncertain systems.

Contribution

It extends the classic particle filter with statistical tests, applying Fisherian and Neyman-Pearson frameworks to improve robustness against malicious or faulty third-party sensor data.

Findings

The proposed filters successfully reject faulty GNSS measurements in simulations.

The Fisher filter performs better when fault models are inaccurate.

The Neyman-Pearson filter excels with reliable fault models.

Abstract

This paper presents a model-based method for fusing data from multiple sensors with a hypothesis-test-based component for rejecting potentially faulty or otherwise malign data. Our framework is based on an extension of the classic particle filter algorithm for real-time state estimation of uncertain systems with nonlinear dynamics with partial and noisy observations. This extension, based on classical statistical theories, utilizes statistical tests against the system's observation model. We discuss the application of the two major statistical testing frameworks, Fisherian significance testing and Neyman-Pearsonian hypothesis testing, to the Monte Carlo and sensor fusion settings. The Monte Carlo Neyman-Pearson test we develop is useful when one has a reliable model of faulty data, while the Fisher one is applicable when one may not have a model of faults, which may occur when dealing with third-party data, like GNSS data of transportation system users. These statistical tests can be combined with a particle filter to obtain a Monte Carlo state estimation scheme that is robust to faulty or outlier data. We present a synthetic freeway traffic state estimation problem where the filters are able to reject simulated faulty GNSS measurements. The fault-model-free Fisher filter, while underperforming the Neyman-Pearson one when the latter has an accurate fault model, outperforms it when the assumed fault model is incorrect.