Optimal Estimation via Nonanticipative Rate Distortion Function and Applications to Time-Varying Gauss-Markov Processes

TL;DR

This paper introduces a causal filtering approach based on nonanticipative rate distortion theory for time-varying Gauss-Markov processes, providing optimal filters that satisfy mean square error constraints and outperform classical Kalman filters in certain scenarios.

Contribution

It develops a novel causal filtering framework using nonanticipative rate distortion theory for time-varying processes, including a reverse-waterfilling algorithm for fidelity constraint satisfaction.

Findings

Derived a universal lower bound on mean square error using conditional mutual information.

Designed optimal filters equivalent to encoder-channel-decoder systems satisfying fidelity constraints.

Demonstrated the approach with illustrative examples showing advantages over classical methods.

Abstract

In this paper, we develop {finite-time horizon} causal filters using the nonanticipative rate distortion theory. We apply the {developed} theory to {design optimal filters for} time-varying multidimensional Gauss-Markov processes, subject to a mean square error fidelity constraint. We show that such filters are equivalent to the design of an optimal \texttt{\{encoder, channel, decoder\}}, which ensures that the error satisfies {a} fidelity constraint. Moreover, we derive a universal lower bound on the mean square error of any estimator of time-varying multidimensional Gauss-Markov processes in terms of conditional mutual information. Unlike classical Kalman filters, the filter developed is characterized by a reverse-waterfilling algorithm, which ensures {that} the fidelity constraint is satisfied. The theoretical results are demonstrated via illustrative examples.