Inexact Adaptive Cubic Regularization Algorithms on Riemannian Manifolds and Application

TL;DR

This paper introduces an inexact adaptive cubic regularization algorithm on Riemannian manifolds, extending previous Euclidean methods, with proven iteration complexity and successful application to joint diagonalization, outperforming trust-region methods.

Contribution

It develops a novel inexact adaptive cubic regularization algorithm on Riemannian manifolds, with complexity analysis and practical application to joint diagonalization.

Findings

Algorithm achieves better performance than trust-region methods.

Provides iteration complexity bounds for approximate second-order optimality.

Successfully applied to joint diagonalization on Stiefel manifold.

Abstract

The adaptive cubic regularization algorithm employing the inexact gradient and Hessian is proposed on general Riemannian manifolds, together with the iteration complexity to get an approximate second-order optimality under certain assumptions on accuracies about the inexact gradient and Hessian. The algorithm extends the inexact adaptive cubic regularization algorithm under true gradient in [Math. Program., 184(1-2): 35-70, 2020] to more general cases even in Euclidean settings. As an application, the algorithm is applied to solve the joint diagonalization problem on the Stiefel manifold. Numerical experiments illustrate that the algorithm performs better than the inexact trust-region algorithm in [Advances of the neural information processing systems, 31, 2018].