Fast Escape, Slow Convergence: Learning Dynamics of Phase Retrieval under Power-Law Data

TL;DR

This paper analyzes the learning dynamics of phase retrieval with power-law spectral data, revealing a three-phase process and deriving explicit scaling laws for convergence and error, supported by experiments.

Contribution

It provides the first rigorous characterization of scaling laws in nonlinear regression with anisotropic data, uncovering how spectral decay influences learning dynamics.

Findings

Identifies a three-phase learning trajectory in anisotropic phase retrieval.

Derives explicit scaling laws linking spectral decay to convergence times.

Experimental validation confirms theoretical predictions of phases and exponents.

Abstract

Scaling laws describe how learning performance improves with data, compute, or training time, and have become a central theme in modern deep learning. We study this phenomenon in a canonical nonlinear model: phase retrieval with anisotropic Gaussian inputs whose covariance spectrum follows a power law. Unlike the isotropic case, where dynamics collapse to a two-dimensional system, anisotropy yields a qualitatively new regime in which an infinite hierarchy of coupled equations governs the evolution of the summary statistics. We develop a tractable reduction that reveals a three-phase trajectory: (i) fast escape from low alignment, (ii) slow convergence of the summary statistics, and (iii) spectral-tail learning in low-variance directions. From this decomposition, we derive explicit scaling laws for the mean-squared error, showing how spectral decay dictates convergence times and error curves. Experiments confirm the predicted phases and exponents. These results provide the first rigorous characterization of scaling laws in nonlinear regression with anisotropic data, highlighting how anisotropy reshapes learning dynamics.