Optimization Methods for Joint Eigendecomposition

TL;DR

This paper develops efficient Hessian-based algorithms for joint eigendecomposition, improving upon existing methods in large-scale applications by enabling better optimization and parameter estimation.

Contribution

It introduces novel algorithms leveraging Hessian structure for joint diagonalization, enhancing efficiency and accuracy over prior approaches.

Findings

Algorithms outperform existing methods in numerical tests.

Hessian-based techniques enable precise parameter estimation.

Methods are scalable to large matrices.

Abstract

Joint diagonalization, the process of finding a shared set of approximate eigenvectors for a collection of matrices, arises in diverse applications such as multidimensional harmonic analysis or quantum information theory. This task is typically framed as an optimization problem: minimizing a non-convex function that quantifies off-diagonal matrix elements across possible bases. In this work, we introduce a suite of efficient algorithms designed to locate local minimizers of this functional. Our methods leverage the Hessian's structure to bypass direct computation of second-order derivatives, evaluating it as either an operator or bilinear form - a strategy that remains computationally feasible even for large-scale applications. Additionally, we demonstrate that this Hessian-based information enables precise estimation of parameters, such as step-size, in first-order optimization techniques like Gradient Descent and Conjugate Gradient, and the design of second-order methods such as (Quasi-)Newton. The resulting algorithms for joint diagonalization outperform existing techniques, and we provide comprehensive numerical evidence of their superior performance.