The Geometry of Algorithms with Orthogonality Constraints

TL;DR

This paper introduces new Newton and conjugate gradient algorithms on Grassmann and Stiefel manifolds, offering a geometric framework that unifies and enhances understanding of algorithms with orthogonality constraints.

Contribution

It develops novel algorithms on key manifolds and provides a geometric theory that unifies and clarifies existing algorithms in numerical linear algebra.

Findings

New Newton and conjugate gradient algorithms for Grassmann and Stiefel manifolds

A geometric framework that offers insights and comparisons of algorithms

A taxonomy that unifies various algorithms in numerical linear algebra

Abstract

In this paper we develop new Newton and conjugate gradient algorithms on the Grassmann and Stiefel manifolds. These manifolds represent the constraints that arise in such areas as the symmetric eigenvalue problem, nonlinear eigenvalue problems, electronic structures computations, and signal processing. In addition to the new algorithms, we show how the geometrical framework gives penetrating new insights allowing us to create, understand, and compare algorithms. The theory proposed here provides a taxonomy for numerical linear algebra algorithms that provide a top level mathematical view of previously unrelated algorithms. It is our hope that developers of new algorithms and perturbation theories will benefit from the theory, methods, and examples in this paper.