Stable Algorithms Lower Bounds for Estimation

TL;DR

This paper establishes that MMSE instability conditions serve as fundamental barriers to stable algorithms in various estimation problems, providing a theoretical framework that links phase transitions to algorithmic limitations.

Contribution

It introduces a criterion based on MMSE instability that explains the failure of stable algorithms and separates them from polynomial-time methods across multiple estimation tasks.

Findings

MMSE instability implies failure of stable algorithms in estimation.

Stable low-degree polynomials cannot solve certain MMSE-unstable problems.

Phase transitions correspond to fundamental algorithmic limits in estimation.

Abstract

In this work, we show that for all statistical estimation problems, a natural MMSE instability (discontinuity) condition implies the failure of stable algorithms, serving as a version of OGP for estimation tasks. Using this criterion, we establish separations between stable and polynomial-time algorithms for the following MMSE-unstable tasks (i) Planted Shortest Path, where Dijkstra's algorithm succeeds, (ii) random Parity Codes, where Gaussian elimination succeeds, and (iii) Gaussian Subset Sum, where lattice-based methods succeed. For all three, we further show that all low-degree polynomials are stable, yielding separations against low-degree methods and a new method to bound the low-degree MMSE. In particular, our technique highlights that MMSE instability is a common feature for Shortest Path and the noiseless Parity Codes and Gaussian subset sum. Last, we highlight that our work places rigorous algorithmic footing on the long-standing physics belief that first-order phase transitions--which in this setting translates to MMSE-instability impose fundamental limits on classes of efficient algorithms.