State-Dependent Lyapunov Method for Rank-1 Matrix Factorization

TL;DR

This paper introduces a state-dependent Lyapunov framework for analyzing gradient descent in rank-1 matrix factorization, providing certificates that guarantee convergence and elucidate dynamics.

Contribution

It develops a novel certificate-based Lyapunov method that explains the convergence behavior of gradient descent in rank-1 matrix factorization.

Findings

Certificates induce monotone dynamics leading to global convergence.

Numerical experiments support the existence of smooth certificate branches.

The method extends to scalar loss functions with quartic augmentation.

Abstract

We study gradient descent for rank-1 matrix factorization through a certificate-based viewpoint. The central object is a parameterized quadratic certificate $I(\delta;\,\cdot)$ whose level sets shrink along the dynamics, thereby inducing a monotone state parameter $\delta_t$. In the certified regime, this mechanism yields convergence to a global minimizer; in the post-critical regime, it forces trajectories toward a terminal balanced manifold. To explain the origin of these certificates, we formulate a state-dependent Lyapunov framework based on structural axioms. Within this framework, the scalar certificate is uniquely determined, and the same local Lagrange analysis constrains the signal and noise blocks of rank-1 extensions. Thus, the certificates arise from the monotonicity structure of the dynamics, rather than from ad hoc algebraic constructions. We also provide numerical evidence beyond the proved cases. For the 2-dimensional rank-1 approximation problem $X=\mathrm{diag}(1,\sigma)$ with $\sigma\in(0,1)$, the experiments are consistent with the existence of a $C^1$ admissible certificate branch. For the quartic-augmented scalar loss $\frac12(ab-1)^2+\mu(ab-1)^4$, the same scalar certificate remains predictive for several values of $\mu$ after choosing an empirical threshold. These experiments suggest that the state-dependent Lyapunov method may extend beyond the settings proved in this paper.