Phase Diagram and Approximate Message Passing for Blind Calibration and Dictionary Learning

TL;DR

This paper analyzes phase transitions in dictionary learning and blind calibration using the replica method, and introduces an approximate message passing algorithm that performs near theoretically optimal in large systems.

Contribution

It provides a theoretical phase diagram for inference feasibility and proposes an AMP algorithm that matches the optimal performance asymptotically.

Findings

Identifies phase transitions delimiting inference regions.

Proposes an AMP algorithm that approaches theoretical limits.

Demonstrates strong empirical performance for calibration tasks.

Abstract

We consider dictionary learning and blind calibration for signals and matrices created from a random ensemble. We study the mean-squared error in the limit of large signal dimension using the replica method and unveil the appearance of phase transitions delimiting impossible, possible-but-hard and possible inference regions. We also introduce an approximate message passing algorithm that asymptotically matches the theoretical performance, and show through numerical tests that it performs very well, for the calibration problem, for tractable system sizes.