Adaptive Localized Cayley Parametrization for Optimization over Stiefel Manifold

TL;DR

This paper introduces an adaptive Cayley parametrization method for optimization on the Stiefel manifold, improving convergence speed by estimating optimal center points and outperforming existing strategies.

Contribution

It proposes an adaptive strategy for Cayley parametrization on the Stiefel manifold, enhancing convergence and efficiency over naive and standard retraction methods.

Findings

Successfully avoids slow convergence in naive Cayley parametrization

Outperforms standard retraction-based strategies in experiments

Provides a unified convergence analysis for the proposed method

Abstract

We present an adaptive parametrization strategy for optimization problems over the Stiefel manifold by using generalized Cayley transforms to utilize powerful Euclidean optimization algorithms efficiently. The generalized Cayley transform can translate an open dense subset of the Stiefel manifold into a vector space, and the open dense subset is determined according to a tunable parameter called a center point. With the generalized Cayley transform, we recently proposed the naive Cayley parametrization, which reformulates the optimization problem over the Stiefel manifold as that over the vector space. Although this reformulation enables us to transplant powerful Euclidean optimization algorithms, their convergences may become slow by a poor choice of center points. To avoid such a slow convergence, in this paper, we propose to estimate adaptively 'good' center points so that the reformulated problem can be solved faster. We also present a unified convergence analysis, regarding the gradient, in cases where fairly standard Euclidean optimization algorithms are employed in the proposed adaptive parametrization strategy. Numerical experiments demonstrate that (i) the proposed strategy succeeds in escaping from the slow convergence observed in the naive Cayley parametrization strategy; (ii) the proposed strategy outperforms the standard strategy which employs a retraction.