On the relation of nonanticipative rate distortion function and filtering theory

TL;DR

This paper explores the connection between nonanticipative rate distortion functions and Bayesian filtering, establishing theoretical foundations and deriving optimal solutions for stationary processes.

Contribution

It formally links nonanticipative RDF with filtering theory, proving existence of optimal distributions and providing closed-form solutions for stationary cases.

Findings

Existence of optimal reproduction distribution established.

Closed-form solution derived for stationary processes.

Illustrative example demonstrating the concepts.

Abstract

In this paper the relation between nonanticipative rate distortion function (RDF) and Bayesian filtering theory is investigated using the topology of weak convergence of probability measures on Polish spaces. The relation is established via an optimization on the space of conditional distributions of the so-called directed information subject to fidelity constraints. Existence of the optimal reproduction distribution of the nonanticipative RDF is shown, while the optimal nonanticipative reproduction conditional distribution for stationary processes is derived in closed form. The realization procedure of nonanticipative RDF which is equivalent to joint-source channel matching for symbol-by-symbol transmission is described, while an example is introduced to illustrate the concepts.