The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices

TL;DR

This paper extends the rigorous proof of the mutual information formula in noisy linear estimation from i.i.d. matrices to a broader class of rotationally invariant matrices using advanced mathematical techniques.

Contribution

It generalizes the existing mutual information formula proof beyond i.i.d. matrices to include a significant class of rotationally invariant matrices.

Findings

Proves the mutual information formula for rotationally invariant matrices.

Utilizes adaptive interpolation and random matrix theory techniques.

Extends previous results from i.i.d. to more general matrix ensembles.

Abstract

There has been definite progress recently in proving the variational single-letter formula given by the heuristic replica method for various estimation problems. In particular, the replica formula for the mutual information in the case of noisy linear estimation with random i.i.d. matrices, a problem with applications ranging from compressed sensing to statistics, has been proven rigorously. In this contribution we go beyond the restrictive i.i.d. matrix assumption and discuss the formula proposed by Takeda, Uda, Kabashima and later by Tulino, Verdu, Caire and Shamai who used the replica method. Using the recently introduced adaptive interpolation method and random matrix theory, we prove this formula for a relevant large sub-class of rotationally invariant matrices.