Optimization Methods for Fully Composite Problems

TL;DR

This paper introduces a unified Fully Composite formulation for convex optimization, encompassing various problem types, and develops natural optimization schemes with proven convergence rates, including accelerated variants for subhomogeneous objectives.

Contribution

It presents a new unified formulation for diverse convex problems and develops natural, convergent optimization schemes with acceleration for specific cases.

Findings

Proposed a Fully Composite formulation covering multiple problem types.

Established global convergence rates for the developed methods.

Designed accelerated variants for subhomogeneous objectives.

Abstract

In this paper, we propose a new Fully Composite Formulation of convex optimization problems. It includes, as a particular case, the problems with functional constraints, max-type minimization problems, and problems of Composite Minimization, where the objective can have simple nondifferentiable components. We treat all these formulations in a unified way, highlighting the existence of very natural optimization schemes of different order. We prove the global convergence rates for our methods under the most general conditions. Assuming that the upper-level component of our objective function is subhomogeneous, we develop efficient modification of the basic Fully Composite first-order and second-order Methods, and propose their accelerated variants.