Information Measures, Inequalities and Performance Bounds for Parameter Estimation in Impulsive Noise Environments

TL;DR

This paper develops new information-theoretic tools and bounds tailored for parameter estimation in impulsive alpha-stable noise environments, where classical Gaussian-based methods fail due to infinite variance.

Contribution

It introduces generalized power and Fisher information measures, extending key inequalities like de Bruijn's identity to alpha-stable noise, enabling performance analysis in heavy-tailed environments.

Findings

Generalized Fisher information and entropy inequalities for alpha-stable noise.

Upper bounds on differential entropy of sums with stable components.

Insights into alpha-stable channel capacity using the new measures.

Abstract

Recent studies found that many channels are affected by additive noise that is impulsive in nature and is best explained by heavy-tailed symmetric alpha-stable distributions. Dealing with impulsive noise environments comes with an added complexity with respect to the standard Gaussian environment: the alpha-stable probability density functions have an infinite second moment and the "nice" Hilbert space structure of the space of random variables having a finite second moment is lost along with its tools and methodologies. This is indeed the case in estimation theory where classical tools to quantify performance of an estimator are tightly related to the assumption of finite variance variables. In alpha-stable environments, expressions such as the mean square error and the Cramer-Rao bound are hence problematic. In this work, we tackle the parameter estimation problem in impulsive noise environments and develop novel tools that are tailored to the alpha-stable and heavy-tailed noise environments, tools that coincide with the standard ones adopted in the Gaussian setup, namely a generalized "power" measure and a generalized Fisher information. We generalize known information inequalities commonly used in the Gaussian context: the de Bruijn's identity, the data processing inequality, the Fisher information inequality, the isoperimetric inequality for entropies and the Cramer-Rao bound. Additionally, we derive upper bounds on the differential entropy of independent sums having a stable component. Finally, the new "power" measure is used to shed some light on the additive alpha-stable noise channel capacity in a setup that generalizes the linear average power constrained AWGN channel. Our theoretical findings are paralleled with numerical evaluations of various quantities and bounds using developed {\em Matlab} packages.