An Exploration of the Role of Principal Inertia Components in Information Theory

TL;DR

This paper investigates how principal inertia components relate to information transfer between variables, revealing their role in estimating functions and maximizing mutual information under symmetry conditions.

Contribution

It introduces a novel interpretation of principal inertia components as filter coefficients and conjectures their equalization maximizes mutual information.

Findings

Principal inertia components act as filter coefficients in estimating functions.

Mutual information is conjectured to be maximized when all principal inertia components are equal.

Results are illustrated with binary string and additive noise channel examples.

Abstract

The principal inertia components of the joint distribution of two random variables $X$ and $Y$ are inherently connected to how an observation of $Y$ is statistically related to a hidden variable $X$. In this paper, we explore this connection within an information theoretic framework. We show that, under certain symmetry conditions, the principal inertia components play an important role in estimating one-bit functions of $X$, namely $f(X)$, given an observation of $Y$. In particular, the principal inertia components bear an interpretation as filter coefficients in the linear transformation of $p_{f(X)|X}$ into $p_{f(X)|Y}$. This interpretation naturally leads to the conjecture that the mutual information between $f(X)$ and $Y$ is maximized when all the principal inertia components have equal value. We also study the role of the principal inertia components in the Markov chain $B\rightarrow X\rightarrow Y\rightarrow \widehat{B}$, where $B$ and $\widehat{B}$ are binary random variables. We illustrate our results for the setting where $X$ and $Y$ are binary strings and $Y$ is the result of sending $X$ through an additive noise binary channel.