Parallel block coordinate descent methods with identification strategies

TL;DR

This paper introduces a parallel block coordinate descent algorithm with identification strategies that improves efficiency in convex optimization by focusing on nonzero coordinates and extending parallelization theory.

Contribution

It develops a parallel variant of a block coordinate descent method incorporating identification strategies, enhancing computational efficiency and extending existing parallelization theory.

Findings

The proposed method converges under specified conditions.

Numerical experiments show improved performance on regression problems.

The strategy effectively identifies nonzero coordinates to focus computation.

Abstract

This work presents a parallel variant of the algorithm introduced in [Acceleration of block coordinate descent methods with identification strategies Comput. Optim. Appl. 72(3):609--640, 2019] to minimize the sum of a partially separable smooth convex function and a possibly non-smooth block-separable convex function under simple constraints. It achieves better efficiency by using a strategy to identify the nonzero coordinates that allows the computational effort to be focused on using a nonuniform probability distribution in the selection of the blocks. Parallelization is achieved by extending the theoretical results from Richt\'arik and Tak\'a\v{c} [Parallel coordinate descent methods for big data optimization, Math. Prog. Ser. A 156:433--484, 2016]. We present convergence results and comparative numerical experiments on regularized regression problems using both synthetic and real data.