Online Learning of Kalman Filtering: From Output to State Estimation

TL;DR

This paper develops online algorithms for learning Kalman filters in partially observed systems, achieving near-optimal regret bounds for output estimation and a trade-off-based approach for state estimation with limited measurements.

Contribution

It introduces a unified online optimization framework for Kalman filtering with unknown models, addressing both output and state estimation, and explores regret bounds under limited query access.

Findings

Achieves $ ext{log } T$ regret for output estimation.

Demonstrates fundamental limitations for state estimation regret.

Proposes a random query scheme enabling $ ext{sqrt } T$ regret with limited measurements.

Abstract

In this paper, we study the problem of learning Kalman filtering with unknown system model in partially observed linear dynamical systems. We propose a unified algorithmic framework based on online optimization that can be used to solve both the output estimation and state estimation scenarios. By exploring the properties of the estimation error cost functions, such as conditionally strong convexity, we show that our algorithm achieves a $\log T$-regret in the horizon length $T$ for the output estimation scenario. More importantly, we tackle the more challenging scenario of learning Kalman filtering for state estimation, which is an open problem in the literature. We first characterize a fundamental limitation of the problem, demonstrating the impossibility of any algorithm to achieve sublinear regret in $T$. By further introducing a random query scheme into our algorithm, we show that a $\sqrt{T}$-regret is achievable when rendering the algorithm limited query access to more informative measurements of the system state in practice. Our algorithm and regret readily capture the trade-off between the number of queries and the achieved regret, and shed light on online learning problems with limited observations. We validate the performance of our algorithms using numerical examples.