Riemannian conditional gradient methods for composite optimization problems

TL;DR

This paper introduces Riemannian conditional gradient algorithms for composite optimization, analyzing their convergence with various step-size strategies and validating performance on manifolds.

Contribution

It develops new Riemannian conditional gradient methods with proven convergence rates for composite functions on manifolds.

Findings

Convergence rate of (1/k) for adaptive and diminishing step sizes

Iteration complexity of (1/psilon^2) for Armijo step size

Numerical experiments on sphere and Stiefel manifolds validate effectiveness

Abstract

In this paper, we propose Riemannian conditional gradient methods for minimizing composite functions, i.e., those that can be expressed as the sum of a smooth function and a retraction-based convex function. We analyze the convergence of the proposed algorithms, utilizing three types of step-size strategies: adaptive, diminishing, and those based on the Armijo condition. We establish the convergence rate of \(\mathcal{O}(1/k)\) for the adaptive and diminishing step sizes, where \(k\) denotes the number of iterations. Additionally, we derive an iteration complexity of \(\mathcal{O}(1/\epsilon^2)\) for the Armijo step-size strategy to achieve \(\epsilon\)-optimality, where \(\epsilon\) is the optimality tolerance. Finally, the effectiveness of our algorithms is validated through some numerical experiments performed on the sphere and Stiefel manifolds.