On convergence of a $q$-random coordinate constrained algorithm for non-convex problems

TL;DR

This paper introduces a flexible random coordinate descent algorithm for non-convex optimization with linear constraints, allowing multiple coordinate updates per iteration, and proves its convergence with improved rates.

Contribution

It presents a novel coordinate descent method that updates an arbitrary number of coordinates simultaneously for non-convex problems, with proven convergence and enhanced convergence rates.

Findings

Convergence of the algorithm is proven.

Updating more coordinates accelerates convergence.

Numerical experiments demonstrate benefits of multiple coordinate updates.

Abstract

We propose a random coordinate descent algorithm for optimizing a non-convex objective function subject to one linear constraint and simple bounds on the variables. Although it is common use to update only two random coordinates simultaneously in each iteration of a coordinate descent algorithm, our algorithm allows updating arbitrary number of coordinates. We provide a proof of convergence of the algorithm. The convergence rate of the algorithm improves when we update more coordinates per iteration. Numerical experiments on large scale instances of different optimization problems show the benefit of updating many coordinates simultaneously.