Principal Inertia Components and Applications

TL;DR

This paper introduces Principal Inertia Components (PICs), a novel framework that decomposes the dependence between discrete variables and links to information theory, estimation, and privacy limits.

Contribution

It provides a new decomposition of dependence via PICs, connecting information theory, estimation, and privacy, with applications to understanding limits of inference and privacy.

Findings

PICs decompose dependence between variables

PICs characterize limits of inference accuracy

PICs relate to privacy constraints

Abstract

We explore properties and applications of the Principal Inertia Components (PICs) between two discrete random variables $X$ and $Y$. The PICs lie in the intersection of information and estimation theory, and provide a fine-grained decomposition of the dependence between $X$ and $Y$. Moreover, the PICs describe which functions of $X$ can or cannot be reliably inferred (in terms of MMSE) given an observation of $Y$. We demonstrate that the PICs play an important role in information theory, and they can be used to characterize information-theoretic limits of certain estimation problems. In privacy settings, we prove that the PICs are related to fundamental limits of perfect privacy.