New results on subgradient methods for strongly convex optimization problems with a unified analysis

TL;DR

This paper introduces a unified framework for subgradient and gradient methods applicable to strongly convex functions, providing optimal algorithms for various problem classes and extending analysis to non-smooth and structured problems.

Contribution

It develops a unifying analysis framework for subgradient methods, leading to optimal algorithms for multiple classes of strongly convex problems, including non-smooth and structured cases.

Findings

Proposes a unifying framework for subgradient and gradient methods.

Yields optimal proximal gradient methods for several problem classes.

Provides nearly optimal conditional gradient methods for smooth and weakly smooth problems.

Abstract

We develop subgradient- and gradient-based methods for minimizing strongly convex functions under a notion which generalizes the standard Euclidean strong convexity. We propose a unifying framework for subgradient methods which yields two kinds of methods, namely, the Proximal Gradient Method (PGM) and the Conditional Gradient Method (CGM), unifying several existing methods. The unifying framework provides tools to analyze the convergence of PGMs and CGMs for non-smooth, (weakly) smooth, and further for structured problems such as the inexact oracle models. The proposed subgradient methods yield optimal PGMs for several classes of problems and yield optimal and nearly optimal CGMs for smooth and weakly smooth problems, respectively.