A Framework of Constraint Preserving Update Schemes for Optimization on Stiefel Manifold

TL;DR

This paper introduces a unified framework for constraint-preserving updates on the Stiefel manifold, proposes a new low-complexity update scheme, and demonstrates its efficiency through numerical experiments on various optimization problems.

Contribution

A novel, unified framework for feasible update schemes on the Stiefel manifold, including a new low-cost scheme and a Barzilai-Borwein-like method with proven convergence.

Findings

The new method is efficient for low-rank correlation matrix problems.

It performs well compared to existing methods like SCF iteration.

Numerical results show improved speed and solution quality.

Abstract

This paper considers optimization problems on the Stiefel manifold $X^{\mathsf{T}}X=I_p$, where $X\in \mathbb{R}^{n \times p}$ is the variable and $I_p$ is the $p$-by-$p$ identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of $X$ and the null space of $X^{\mathsf{T}}$. While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai-Borwein-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn-Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn-Sham total energy minimization.