Optimization flows landing on the Stiefel manifold

TL;DR

This paper introduces a continuous-time optimization flow that asymptotically converges to the Stiefel manifold, extending the canonical metric to a generalized version and analyzing stability and convergence properties.

Contribution

It proposes a novel flow on a generalized Stiefel manifold that guarantees global convergence and stability of critical points, extending existing Riemannian optimization methods.

Findings

Flow asymptotically lands on the Stiefel manifold

Global convergence to critical points is proven

Local minima and isolated critical points are stable

Abstract

We study a continuous-time system that solves optimization problems over the set of orthonormal matrices, which is also known as the Stiefel manifold. The resulting optimization flow follows a path that is not always on the manifold but asymptotically lands on the manifold. We introduce a generalized Stiefel manifold to which we extend the canonical metric of the Stiefel manifold. We show that the vector field of the proposed flow can be interpreted as the sum of a Riemannian gradient on a generalized Stiefel manifold and a normal vector. Moreover, we prove that the proposed flow globally converges to the set of critical points, and any local minimum and isolated critical point is asymptotically stable.