Accelerated Directional Search with non-Euclidean prox-structure

TL;DR

This paper introduces an accelerated directional search method utilizing non-Euclidean prox-structure, improving the efficiency of Nesterov's method for certain convex optimization problems by leveraging linear coupling and norm-specific steps.

Contribution

It presents a novel accelerated directional search algorithm that combines linear coupling with mixed norm steps, achieving up to n-fold speedup for convex unconstrained problems.

Findings

Method is n-times faster under certain norm conditions.

Approach remains stable in constrained optimization scenarios.

Utilizes linear coupling to enhance acceleration.

Abstract

In the paper we propose an accelerated directional search method with non-euclidian prox-structure. We consider convex unconstraint optimization problem in $\mathbb{R}^n$. For simplicity we start from the zero point. We expect in advance that 1-norm of the solution is close enough to its 2-norm. In this case the standard accelerated Nesterov's directional search method can be improved. In the paper we show how to make Nesterov's method $n$-times faster (up to a $\log n$-factor) in this case. The basic idea is to use linear coupling, proposed by Allen-Zhu & Orecchia in 2014, and to make Grad-step in 2-norm, but Mirr-step in 1-norm. We show that for constrained optimization problems this approach stable upon an obstacle.