A Riemannian Proximal Newton-CG Method

TL;DR

This paper introduces a Riemannian proximal Newton-CG method that combines conjugate gradient techniques with Newton methods, ensuring global convergence and superlinear local convergence for optimization problems on manifolds, and demonstrates superior performance in experiments.

Contribution

It proposes a novel Riemannian proximal Newton-CG algorithm that guarantees both global and local superlinear convergence, improving upon existing hybrid approaches.

Findings

The method converges globally and superlinearly locally.

Numerical experiments show it outperforms existing methods.

The approach effectively handles optimization on manifolds with sparsity constraints.

Abstract

Recently, a Riemannian proximal Newton method has been developed for optimizing problems in the form of $\min_{x\in\mathcal{M}} f(x) + \mu \|x\|_1$, where $\mathcal{M}$ is a compact embedded submanifold and $f(x)$ is smooth. Although this method converges superlinearly locally, global convergence is not guaranteed. The existing remedy relies on a hybrid approach: running a Riemannian proximal gradient method until the iterate is sufficiently accurate and switching to the Riemannian proximal Newton method. This existing approach is sensitive to the switching parameter. This paper proposes a Riemannian proximal Newton-CG method that merges the truncated conjugate gradient method with the Riemannian proximal Newton method. The global convergence and local superlinear convergence are proven. Numerical experiments show that the proposed method outperforms other state-of-the-art methods.