Data processing inequalities based on a certain structured class of information measures with application to estimation theory

TL;DR

This paper introduces a new class of information measures based on nested convex functions and likelihood ratios, extending the Bhattacharyya distance, and applies them to derive tighter bounds in estimation problems, especially under channel uncertainty.

Contribution

The paper develops a novel class of data processing inequalities using structured information measures that generalize the Bhattacharyya distance, with applications to estimation bounds under noise and fading.

Findings

Tighter lower bounds for parameter estimation in AWGN channels.

Enhanced bounds under channel uncertainty like fading.

Demonstrated the convexity property influences bounds in uncertain channels.

Abstract

We study data processing inequalities that are derived from a certain class of generalized information measures, where a series of convex functions and multiplicative likelihood ratios are nested alternately. While these information measures can be viewed as a special case of the most general Zakai-Ziv generalized information measure, this special nested structure calls for attention and motivates our study. Specifically, a certain choice of the convex functions leads to an information measure that extends the notion of the Bhattacharyya distance (or the Chernoff divergence): While the ordinary Bhattacharyya distance is based on the (weighted) geometric mean of two replicas of the channel's conditional distribution, the more general information measure allows an arbitrary number of such replicas. We apply the data processing inequality induced by this information measure to a detailed study of lower bounds of parameter estimation under additive white Gaussian noise (AWGN) and show that in certain cases, tighter bounds can be obtained by using more than two replicas. While the resulting lower bound may not compete favorably with the best bounds available for the ordinary AWGN channel, the advantage of the new lower bound, relative to the other bounds, becomes significant in the presence of channel uncertainty, like unknown fading. This different behavior in the presence of channel uncertainty is explained by the convexity property of the information measure.