An optimal subgradient algorithm for large-scale convex optimization in simple domains

TL;DR

This paper demonstrates that the optimal subgradient algorithm (OSGA) effectively solves large-scale convex optimization problems with simple domains, achieving optimal complexity bounds using only first-order information.

Contribution

It extends OSGA to structured large-scale convex problems with simple domains or constraints, providing efficient solutions and practical implementation details.

Findings

Achieves optimal complexity bounds for smooth and nonsmooth problems.

Provides efficient solution methods for large-scale convex problems.

Demonstrates practical efficiency through numerical experiments.

Abstract

This paper shows that the optimal subgradient algorithm, OSGA, proposed in \cite{NeuO} can be used for solving structured large-scale convex constrained optimization problems. Only first-order information is required, and the optimal complexity bounds for both smooth and nonsmooth problems are attained. More specifically, we consider two classes of problems: (i) a convex objective with a simple closed convex domain, where the orthogonal projection on this feasible domain is efficiently available; (ii) a convex objective with a simple convex functional constraint. If we equip OSGA with an appropriate prox-function, the OSGA subproblem can be solved either in a closed form or by a simple iterative scheme, which is especially important for large-scale problems. We report numerical results for some applications to show the efficiency of the proposed scheme. A software package implementing OSGA for above domains is available.