Global Optimization with Orthogonality Constraints via Stochastic Diffusion on Manifold

TL;DR

This paper introduces a stochastic diffusion approach on the Stiefel manifold to efficiently find global solutions in orthogonality constrained optimization problems, with proven convergence and demonstrated effectiveness on diverse applications.

Contribution

It develops a novel stochastic differential equation-based scheme on the Stiefel manifold with convergence guarantees for global optimization under orthogonality constraints.

Findings

Proposed method converges to global minimizers under certain conditions.

Effective in applications like polynomial optimization and Cryo-EM structure determination.

Numerical scheme based on Cayley transformation is computationally efficient.

Abstract

Orthogonality constrained optimization is widely used in applications from science and engineering. Due to the nonconvex orthogonality constraints, many numerical algorithms often can hardly achieve the global optimality. We aim at establishing an efficient scheme for finding global minimizers under one or more orthogonality constraints. The main concept is based on noisy gradient flow constructed from stochastic differential equations (SDE) on the Stiefel manifold, the differential geometric characterization of orthogonality constraints. We derive an explicit representation of SDE on the Stiefel manifold endowed with a canonical metric and propose a numerically efficient scheme to simulate this SDE based on Cayley transformation with theoretical convergence guarantee. The convergence to global optimizers is proved under second-order continuity. The effectiveness and efficiency of the proposed algorithms are demonstrated on a variety of problems including homogeneous polynomial optimization, computation of stability number, and 3D structure determination from Common Lines in Cryo-EM.