Metric and topological entropy bounds for optimal coding of stochastic dynamical systems

TL;DR

This paper establishes bounds on the minimal communication capacity needed for optimal zero-delay coding and estimation of stochastic dynamical systems, linking entropy measures with control and information theory.

Contribution

It introduces new bounds based on topological and metric entropy, connecting dynamical systems with information-theoretic approaches for stochastic control.

Findings

Lower bounds on channel capacity for different stability criteria

Memoryless noisy channels do not hinder asymptotic estimation in noiseless systems

New methods connect dynamical systems, control, and information theory

Abstract

We consider the problem of optimal zero-delay coding and estimation of a stochastic dynamical system over a noisy communication channel under three estimation criteria concerned with the low-distortion regime. The criteria considered are (i) a strong and (ii) a weak form of almost sure stability of the estimation error as well as (ii) quadratic stability in expectation. For all three objectives, we derive lower bounds on the smallest channel capacity $C_0$ above which the objective can be achieved with an arbitrarily small error. We first obtain bounds through a dynamical systems approach by constructing an infinite-dimensional dynamical system and relating the capacity with the topological and the metric entropy of this dynamical system. We also consider information-theoretic and probability-theoretic approaches to address the different criteria. Finally, we prove that a memoryless noisy channel in general constitutes no obstruction to asymptotic almost sure state estimation with arbitrarily small errors, when there is no noise in the system. The results provide new solution methods for the criteria introduced (e.g., standard information-theoretic bounds cannot be applied for some of the criteria) and establish further connections between dynamical systems, networked control, and information theory, and especially in the context of nonlinear stochastic systems.