Information and estimation in Fokker-Planck channels

TL;DR

This paper explores the connection between information theory and estimation in systems described by Fokker-Planck equations, extending classical identities to more general stochastic differential equations.

Contribution

It generalizes De Bruijn's identity and the I-MMSE relation to Fokker-Planck channels, linking entropy derivatives to estimation-theoretic quantities.

Findings

Derived new identities relating entropy and Fisher information in Fokker-Planck systems

Extended classical information-estimation relations to stochastic differential equations

Provided a framework for analyzing time-evolving information measures

Abstract

We study the relationship between information- and estimation-theoretic quantities in time-evolving systems. We focus on the Fokker-Planck channel defined by a general stochastic differential equation, and show that the time derivatives of entropy, KL divergence, and mutual information are characterized by estimation-theoretic quantities involving an appropriate generalization of the Fisher information. Our results vastly extend De Bruijn's identity and the classical I-MMSE relation.