Riemannian Optimization over Symmetric Positive Definite Matrices with the Alpha-Procrustes Geometry

TL;DR

This paper introduces the Alpha-Procrustes geometry on the SPD manifold, which ensures uniformly bounded eigenvalues of the Riemannian Hessian, improving optimization robustness for ill-conditioned matrices.

Contribution

It proposes the Alpha-Procrustes geometry with lpha=1, providing a robust Riemannian framework with eigenvalue bounds independent of matrix conditioning.

Findings

Eigenvalues of the Riemannian metric operator are uniformly bounded for lpha=1.

Under Euclidean Hessian spectral bounds, Riemannian Hessian eigenvalues are also bounded independently of matrix condition.

Numerical experiments confirm the theoretical robustness of the proposed geometry.

Abstract

In Riemannian optimization, it is well known that the condition number of the Riemannian Hessian at an optimum strongly influences the asymptotic convergence behavior of optimization algorithms. On the manifold of symmetric positive definite (SPD) matrices, several commonly used metrics for optimization, such as the Affine-Invariant (AI) and Bures--Wasserstein (BW) metrics, tend to become ill-conditioned as the underlying SPD matrix becomes ill-conditioned. As a result, even when the Euclidean Hessian remains uniformly well-conditioned on the SPD manifold, optimization may still become difficult near an optimum associated with an ill-conditioned SPD matrix. In this paper, we address this issue through the Alpha-Procrustes (AP) geometry on the SPD manifold. This geometry generalizes several well-known metrics, including the Log-Euclidean (LE) metric for \(\alpha=0\) and the BW metric for \(\alpha=1/2\). We first show that, when \(\alpha=1\), all eigenvalues of the Riemannian metric operator induced by the AP geometry are uniformly bounded independently of the underlying SPD matrix. Therefore, under the assumption that the Euclidean Hessian satisfies the uniform spectral bounds, all the eigenvalues of the corresponding Riemannian Hessian are uniformly bounded independently of the underlying SPD matrix. Consequently, the case \(\alpha=1\) provides a robust geometric framework for several Riemannian optimization problems involving ill-conditioned SPD matrices. Finally, we validate our theoretical findings through extensive numerical experiments across a range of applications.