Split-Merge Revisited: A Scalable Approach to Generalized Eigenvalue Problems

TL;DR

This paper introduces a scalable, stable, and efficient approach to solving generalized eigenvalue problems by reformulating the problem, establishing convergence guarantees, and extending the Split-Merge algorithm with second-order information, demonstrating superior performance.

Contribution

It proposes a difference-based formulation of GEP, develops an accelerated preconditioned mirror descent algorithm, and extends the Split-Merge method to improve convergence and stability in large-scale problems.

Findings

Significant improvements in computational efficiency over existing methods.

Enhanced numerical stability in large-scale GEP solutions.

Empirical validation on synthetic and real datasets confirms effectiveness.

Abstract

The generalized eigenvalue problem (GEP) serves as a cornerstone in a wide range of applications in numerical linear algebra and scientific computing. However, traditional approaches that aim to maximize the classical Rayleigh quotient often suffer from numerical instability and limited computational efficiency, especially in large-scale settings. In this work, we explore an alternative difference-based formulation of GEP by minimizing a structured quadratic polynomial objective, which enables the application of efficient first-order optimization methods. We establish global convergence guarantees for these methods without requiring line search, and further introduce a transform-domain perspective that reveals the intrinsic connection and performance gap between classical first-order algorithms and the power method. Based on this insight, we develop an accelerated preconditioned mirror descent algorithm, which allows for flexible preconditioner design and improved convergence behavior. Lastly, we extend the recently proposed Split-Merge algorithm to the general GEP setting, incorporating richer second-order information to further accelerate convergence. Empirical results on both synthetic and real-world datasets demonstrate that our proposed methods achieve significant improvements over existing baselines in terms of both computational efficiency and numerical stability.