Some Results on the Information Loss in Dynamical Systems

TL;DR

This paper investigates the limits of information preservation in nonlinear dynamical systems, providing bounds on information loss and identifying systems with negligible information loss, including certain finite-precision implementations.

Contribution

It introduces an upper bound on information loss rate and characterizes a family of systems, including nonlinear and finite-precision systems, with vanishing information loss.

Findings

Linear filters have zero information loss.

Certain nonlinear and finite-precision systems also exhibit negligible information loss.

An upper bound on the information loss rate is established.

Abstract

In this work we investigate the information loss in (nonlinear) dynamical input-output systems and provide some general results. In particular, we present an upper bound on the information loss rate, defined as the (non-negative) difference between the entropy rates of the jointly stationary stochastic processes at the input and output of the system. We further introduce a family of systems with vanishing information loss rate. It is shown that not only linear filters belong to that family, but - under certain circumstances - also finite-precision implementations of the latter, which typically consist of nonlinear elements.