Landscape Correspondence of Empirical and Population Risks in the Eigendecomposition Problem

TL;DR

This paper explores the geometric landscape of eigendecomposition problems in spectral methods, establishing connections between empirical and population risks, and applies findings to low-rank matrix optimization with supporting simulations.

Contribution

It extends landscape analysis near critical points for eigenvector problems and introduces a novel link between empirical and true data matrix landscapes in eigendecomposition.

Findings

Extended landscape analysis near critical points.

Established a connection between empirical and population risk landscapes.

Validated theoretical results through simulations.

Abstract

Spectral methods include a family of algorithms related to the eigenvectors of certain data-generated matrices. In this work, we are interested in studying the geometric landscape of the eigendecomposition problem in various spectral methods. In particular, we first extend known results regarding the landscape at critical points to larger regions near the critical points in a special case of finding the leading eigenvector of a symmetric matrix. For a more general eigendecomposition problem, inspired by recent findings on the connection between the landscapes of empirical risk and population risk, we then build a novel connection between the landscape of an eigendecomposition problem that uses random measurements and the one that uses the true data matrix. We also apply our theory to a variety of low-rank matrix optimization problems and conduct a series of simulations to illustrate our theoretical findings.