A homogeneous Rayleigh quotient with applications in gradient methods

TL;DR

The paper introduces a new homogeneous Rayleigh quotient, analyzes its sensitivity, and explores its application as a stepsize in gradient methods for optimization, extending to generalized eigenvalue problems.

Contribution

It proposes a novel homogeneous Rayleigh quotient, analyzes its properties, and applies it as a stepsize in gradient methods, extending the concept to generalized eigenvalue problems.

Findings

Homogeneous Rayleigh quotient has well-defined sensitivity properties.

Application as stepsize improves gradient method performance.

Extension of the quotient concept to generalized eigenvalue problems.

Abstract

Given an approximate eigenvector, its (standard) Rayleigh quotient and harmonic Rayleigh quotient are two well-known approximations of the corresponding eigenvalue. We propose a new type of Rayleigh quotient, the homogeneous Rayleigh quotient, and analyze its sensitivity with respect to perturbations in the eigenvector. Furthermore, we study the inverse of this homogeneous Rayleigh quotient as stepsize for the gradient method for unconstrained optimization. The notion and basic properties are also extended to the generalized eigenvalue problem.