Maximum entropy low-rank matrix recovery

TL;DR

This paper introduces MaxEnt, an information-theoretic method for low-rank matrix recovery that maximizes observation entropy to improve data acquisition efficiency, outperforming random measurements in simulations and real-world applications.

Contribution

The paper presents a novel MaxEnt approach that connects information-theoretic sampling with subspace packings and provides efficient measurement mask algorithms for matrix recovery.

Findings

MaxEnt significantly outperforms random measurements in simulations.

Effective in real-world applications like image recovery and text indexing.

Reveals new links between entropy maximization and subspace packing strategies.

Abstract

We propose in this paper a novel, information-theoretic method, called MaxEnt, for efficient data acquisition for low-rank matrix recovery. This proposed method has important applications to a wide range of problems, including image processing and text document indexing. Fundamental to our design approach is the so-called maximum entropy principle, which states that the measurement masks which maximize the entropy of observations, also maximize the information gain on the unknown matrix $\mathbf{X}$. Coupled with a low-rank stochastic model for $\mathbf{X}$, such a principle (i) reveals novel connections between information-theoretic sampling and subspace packings, and (ii) yields efficient mask construction algorithms for matrix recovery, which significantly outperforms random measurements. We illustrate the effectiveness of MaxEnt in simulation experiments, and demonstrate its usefulness in two real-world applications on image recovery and text document indexing.