Universal and Parameter-free Gradient Sliding for Composite Optimization

TL;DR

This paper introduces PFUGS, a novel parameter-free gradient sliding algorithm for convex composite optimization that efficiently handles unknown problem constants.

Contribution

The paper presents the first gradient sliding method applicable to problems with two functions having unknown constants, without requiring prior parameter knowledge.

Findings

PFUGS computes ε-approximate solutions with optimal gradient evaluations.

PFUGS does not require prior knowledge of problem constants.

Achieves convergence rates matching known bounds for composite optimization.

Abstract

We propose a Parameter-Free Universal Gradient Sliding (PFUGS) algorithm for computing an approximate solution to the convex composite optimization $\min_{x\in X} \{f(x) + g(x)\}$, where $f$ has $(M_\nu,\nu)$-H\"older continuous subgradient and $g$ has $L$-Lipschitz continuous gradient. PFUGS computes an $\varepsilon$-approximate solution with $\mathcal{O}((M_\nu/\varepsilon)^{{2}/{(1+3\nu)}})$ evaluations of (sub)gradients of $f$ and $\mathcal{O}((L/\varepsilon)^{1/2})$ evaluations of gradients of $g$, without prior knowledge of problem constants. To the best of our knowledge, PFUGS is the first gradient sliding algorithm for problems involving two functions whose distinct problem constants are both unknown a priori.