Accelerated Proximal Envelopes: Application to the Coordinate Descent Method

TL;DR

This paper introduces a proximally accelerated coordinate descent method that leverages universal accelerated proximal envelopes to improve computational efficiency and exploit problem sparsity, especially for high-dimensional optimization tasks.

Contribution

It proposes a novel proximally accelerated coordinate descent algorithm that reduces complexity dependence on problem dimension and demonstrates practical faster convergence.

Findings

Achieves $ ilde{O}(rac{1}{ oot n})$ complexity for SoftMax-like functions.

Demonstrates faster convergence compared to standard methods.

Effectively exploits problem sparsity.

Abstract

This article is devoted to one particular case of using universal accelerated proximal envelopes to obtain computationally efficient accelerated versions of methods used to solve various optimization problem setups. In this paper, we propose a proximally accelerated coordinate descent method that achieves the efficient algorithmic complexity of iteration and allows one to take advantage of the problem sparseness. An example of applying the proposed approach to optimizing a SoftMax-like function considered, for which the described method allowing weaken the dependence of the computational complexity on the dimension of the problem $n$ in $\mathcal{O}(\sqrt{n})$ times, and in practice demonstrates a faster convergence in comparison with standard methods.