Adaptive Riemannian ADMM for Nonsmooth Optimization: Optimal Complexity without Smoothing

TL;DR

This paper introduces an adaptive Riemannian ADMM method that achieves optimal convergence rates for nonsmooth optimization on manifolds without smoothing, demonstrated through applications like sparse PCA.

Contribution

It presents the first Riemannian ADMM that converges optimally without smoothing the nonsmooth term, using only one gradient and proximal update per iteration.

Findings

Achieves optimal complexity of O(ε^{-3}) for ε-approximate KKT points

Outperforms existing Riemannian ADMM variants in experiments

Effective on applications like sparse PCA and robust subspace recovery

Abstract

We study the problem of minimizing the sum of a smooth function and a nonsmooth convex regularizer over a compact Riemannian submanifold embedded in Euclidean space. By introducing an auxiliary splitting variable, we propose an adaptive Riemannian alternating direction method of multipliers (ARADMM), which, for the first time, achieves convergence without requiring smoothing of the nonsmooth term. Our approach involves only one Riemannian gradient evaluation and one proximal update per iteration. Through careful and adaptive coordination of the stepsizes and penalty parameters, we establish an optimal iteration complexity of order $\mathcal{O}(\epsilon^{-3})$ for finding an $\epsilon$-approximate KKT point, matching the complexity of existing smoothing technique-based Riemannian ADMM methods. Extensive numerical experiments on sparse PCA and robust subspace recovery demonstrate that our ARADMM consistently outperforms state-of-the-art Riemannian ADMM variants in convergence speed and solution quality.