A Universal Primal-Dual Convex Optimization Framework

TL;DR

This paper introduces a universal primal-dual convex optimization framework that adapts to unknown Holder smoothness, avoids complex operators, and guarantees optimal convergence rates for constrained convex problems.

Contribution

The framework is novel in its universality, adaptability to unknown smoothness, and avoidance of proximity operators, extending applicability to non-differentiable objectives and general linear constraints.

Findings

Achieves optimal convergence rates for various Holder smoothness degrees.

Does not require the objective function to be differentiable.

Handles additional general linear inclusion constraints.

Abstract

We propose a new primal-dual algorithmic framework for a prototypical constrained convex optimization template. The algorithmic instances of our framework are universal since they can automatically adapt to the unknown Holder continuity degree and constant within the dual formulation. They are also guaran- teed to have optimal convergence rates in the objective residual and the feasibility gap for each Holder smoothness degree. In contrast to existing primal-dual algorithms, our framework avoids the proximity operator of the objective function. We instead leverage computationally cheaper, Fenchel-type operators, which are the main workhorses of the generalized conditional gradient (GCG)-type methods. In contrast to the GCG-type methods, our framework does not require the objective function to be differentiable, and can also process additional general linear inclusion constraints, while guarantees the convergence rate on the primal problem