Generalized Eigenvalue Problems with Generative Priors

TL;DR

This paper investigates generalized eigenvalue problems with generative priors, proposing an algorithm that converges linearly to optimal solutions, supported by theoretical analysis and numerical experiments.

Contribution

It introduces the Projected Rayleigh Flow Method (PRFM) for GEPs with generative priors and proves its linear convergence to optimal solutions under certain conditions.

Findings

PRFM converges linearly to the optimal solution.

The method achieves the optimal statistical rate.

Numerical results confirm the effectiveness of PRFM.

Abstract

Generalized eigenvalue problems (GEPs) find applications in various fields of science and engineering. For example, principal component analysis, Fisher's discriminant analysis, and canonical correlation analysis are specific instances of GEPs and are widely used in statistical data processing. In this work, we study GEPs under generative priors, assuming that the underlying leading generalized eigenvector lies within the range of a Lipschitz continuous generative model. Under appropriate conditions, we show that any optimal solution to the corresponding optimization problems attains the optimal statistical rate. Moreover, from a computational perspective, we propose an iterative algorithm called the Projected Rayleigh Flow Method (PRFM) to approximate the optimal solution. We theoretically demonstrate that under suitable assumptions, PRFM converges linearly to an estimated vector that achieves the optimal statistical rate. Numerical results are provided to demonstrate the effectiveness of the proposed method.