Recurrent Neural Networks and Universal Approximation of Bayesian Filters

TL;DR

This paper explores how recurrent neural networks can learn to perform Bayesian filtering directly from data, providing theoretical guarantees on approximation accuracy and long-term performance.

Contribution

It introduces a generic RNN framework for Bayesian filtering and establishes approximation error bounds, including guarantees for long-time accuracy.

Findings

Provides approximation error bounds for RNN-based filters

Establishes time-uniform error bounds for long-term filtering

Discusses practical implications of theoretical results

Abstract

We consider the Bayesian optimal filtering problem: i.e. estimating some conditional statistics of a latent time-series signal from an observation sequence. Classical approaches often rely on the use of assumed or estimated transition and observation models. Instead, we formulate a generic recurrent neural network framework and seek to learn directly a recursive mapping from observational inputs to the desired estimator statistics. The main focus of this article is the approximation capabilities of this framework. We provide approximation error bounds for filtering in general non-compact domains. We also consider strong time-uniform approximation error bounds that guarantee good long-time performance. We discuss and illustrate a number of practical concerns and implications of these results.