Operator-valued formulas for Riemannian Gradient and Hessian and families of tractable metrics

TL;DR

This paper derives explicit formulas for Riemannian gradient and Hessian on quotient manifolds with non-constant metrics, enabling new optimization methods on classical and positive-semidefinite matrix manifolds.

Contribution

It provides explicit formulas for Riemannian connections and Hessians for quotient manifolds with non-constant metrics, expanding optimization tools.

Findings

Formulas for Riemannian gradient and Hessian on various manifolds

New family of metrics on positive-semidefinite matrices

Frameworks for Riemannian optimization on quotient manifolds

Abstract

We provide an explicit formula for the Levi-Civita connection and Riemannian Hessian for a Riemannian manifold that is a quotient of a manifold embedded in an inner product space with a non-constant metric function. Together with a classical formula for projection, this allows us to evaluate Riemannian gradient and Hessian for several families of metrics on classical manifolds, including a family of metrics on Stiefel manifolds connecting both the constant and canonical ambient metrics with closed-form geodesics. Using these formulas, we derive Riemannian optimization frameworks on quotients of Stiefel manifolds, including flag manifolds, and a new family of complete quotient metrics on the manifold of positive-semidefinite matrices of fixed rank, considered as a quotient of a product of Stiefel and positive-definite matrix manifold with affine-invariant metrics. The method is procedural, and in many instances, the Riemannian gradient and Hessian formulas could be derived by symbolic calculus. The method extends the list of potential metrics that could be used in manifold optimization and machine learning.