Mutual Information in Rank-One Matrix Estimation

TL;DR

This paper establishes bounds on the mutual information for rank-one matrix estimation problems using an interpolation method, revealing phase transitions and extending to low-rank tensor estimation.

Contribution

It provides the first rigorous bounds on mutual information for a broad class of rank-one matrix estimation problems, confirming conjectures and extending to low-rank tensor cases.

Findings

Bounds match in a large parameter region

Existence of a phase transition where spectrum is uninformative

Method applicable to low-rank tensor estimation

Abstract

We consider the estimation of a n-dimensional vector x from the knowledge of noisy and possibility non-linear element-wise measurements of xxT , a very generic problem that contains, e.g. stochastic 2-block model, submatrix localization or the spike perturbation of random matrices. We use an interpolation method proposed by Guerra and later refined by Korada and Macris. We prove that the Bethe mutual information (related to the Bethe free energy and conjectured to be exact by Lesieur et al. on the basis of the non-rigorous cavity method) always yields an upper bound to the exact mutual information. We also provide a lower bound using a similar technique. For concreteness, we illustrate our findings on the sparse PCA problem, and observe that (a) our bounds match for a large region of parameters and (b) that it exists a phase transition in a region where the spectum remains uninformative. While we present only the case of rank-one symmetric matrix estimation, our proof technique is readily extendable to low-rank symmetric matrix or low-rank symmetric tensor estimation