Generalized Eigenvalue Problems with Generative Priors
Zhaoqiang Liu, Wen Li, Junren Chen

TL;DR
This paper investigates generalized eigenvalue problems with generative priors, proposing an algorithm that converges linearly to optimal solutions, supported by theoretical analysis and numerical experiments.
Contribution
It introduces the Projected Rayleigh Flow Method (PRFM) for GEPs with generative priors and proves its linear convergence to optimal solutions under certain conditions.
Findings
PRFM converges linearly to the optimal solution.
The method achieves the optimal statistical rate.
Numerical results confirm the effectiveness of PRFM.
Abstract
Generalized eigenvalue problems (GEPs) find applications in various fields of science and engineering. For example, principal component analysis, Fisher's discriminant analysis, and canonical correlation analysis are specific instances of GEPs and are widely used in statistical data processing. In this work, we study GEPs under generative priors, assuming that the underlying leading generalized eigenvector lies within the range of a Lipschitz continuous generative model. Under appropriate conditions, we show that any optimal solution to the corresponding optimization problems attains the optimal statistical rate. Moreover, from a computational perspective, we propose an iterative algorithm called the Projected Rayleigh Flow Method (PRFM) to approximate the optimal solution. We theoretically demonstrate that under suitable assumptions, PRFM converges linearly to an estimated vector that…
Peer Reviews
Decision·NeurIPS 2024 poster
Generalized eigenproblems are a cornerstone of many ML problems and their study could have a great impact. The idea of constraining the solution with the output of a generative model (although a bit unclear) has a good potential.
The presentation of the problem could be improved (for exemple, it is never said that Eq 3 is a Rayleigh quotient), section 2.2 gather folklore results that are not useful (in my opinion). It is also difficult to follow how the algorithm 1 is derived from section 3. The numerical experiments are carried out on toy data and it makes it difficult to understand how it could be used in practice.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMatrix Theory and Algorithms · Stability and Control of Uncertain Systems
