Unifying Framework for Accelerated Randomized Methods in Convex Optimization

TL;DR

This paper introduces a unifying framework for developing accelerated randomized methods in convex optimization, accommodating inexact oracle information and various problem structures, with theoretical convergence guarantees.

Contribution

The paper presents a comprehensive framework that unifies and extends accelerated randomized methods for convex optimization with inexact oracles and diverse constraints.

Findings

Developed accelerated random block-coordinate descent with inexact oracle

Created derivative-free accelerated methods for convex problems

Extended the framework to strongly convex and inexact model scenarios

Abstract

In this paper, we consider smooth convex optimization problems with simple constraints and inexactness in the oracle information such as value, partial or directional derivatives of the objective function. We introduce a unifying framework, which allows to construct different types of accelerated randomized methods for such problems and to prove convergence rate theorems for them. We focus on accelerated random block-coordinate descent, accelerated random directional search, accelerated random derivative-free method and, using our framework, provide their versions for problems with inexact oracle information. Our contribution also includes accelerated random block-coordinate descent with inexact oracle and entropy proximal setup as well as derivative-free version of this method. Moreover, we present an extension of our framework for strongly convex optimization problems. We also discuss an extension for the case of inexact model of the objective function.