Learning the Kalman Filter with Fine-Grained Sample Complexity

TL;DR

This paper introduces a model-free policy gradient method for Kalman filtering, achieving near-optimal sample complexity without prior system stability assumptions, advancing control in noisy and disturbed linear systems.

Contribution

It presents the first end-to-end sample complexity analysis for model-free policy gradient methods in Kalman filtering, introducing the receding-horizon framework without requiring system stability or prior filters.

Findings

Achieves $ ilde{\mathcal{O}}(\epsilon^{-2})$ sample complexity for learning stabilizing filters.

Does not require open-loop stability or prior stabilizing filters.

Applicable to systems with noisy and adversarial disturbances.

Abstract

We develop the first end-to-end sample complexity of model-free policy gradient (PG) methods in discrete-time infinite-horizon Kalman filtering. Specifically, we introduce the receding-horizon policy gradient (RHPG-KF) framework and demonstrate $\tilde{\mathcal{O}}(\epsilon^{-2})$ sample complexity for RHPG-KF in learning a stabilizing filter that is $\epsilon$-close to the optimal Kalman filter. Notably, the proposed RHPG-KF framework does not require the system to be open-loop stable nor assume any prior knowledge of a stabilizing filter. Our results shed light on applying model-free PG methods to control a linear dynamical system where the state measurements could be corrupted by statistical noises and other (possibly adversarial) disturbances.