Stochastic Zeroth-Order Method for Computing Generalized Rayleigh Quotients

TL;DR

This paper introduces a stochastic zeroth-order Riemannian algorithm for maximizing the generalized Rayleigh quotient that avoids matrix-adjoint products, providing convergence guarantees and demonstrating superior performance in experiments.

Contribution

It presents a novel zeroth-order method that does not require adjoint computations for Rayleigh quotient maximization, with proven convergence guarantees.

Findings

Converges to global maximizers at a sublinear rate with probability one.

Outperforms state-of-the-art algorithms in numerical experiments.

Avoids errors from adjoint mismatches in matrix computations.

Abstract

The maximization of the (generalized) Rayleigh quotient is a central problem in numerical linear algebra. Conventional algorithms for its computation typically rely on matrix-adjoint products, making them sensitive to errors arising from adjoint mismatches. To address this issue, we introduce a stochastic zeroth-order Riemannian algorithm that maximizes the generalized Rayleigh quotient without requiring adjoint or matrix inverse computations. We provide theoretical convergence guarantees showing that the iterates converge to the set of global maximizers of the (generalized) Rayleigh quotient at a sublinear rate with probability one. Our theoretical results are supported by numerical experiments, which demonstrate the excellent performance of the proposed method compared to state-of-the-art algorithms.