Faster Convergence of a Randomized Coordinate Descent Method for Linearly Constrained Optimization Problems

TL;DR

This paper improves the convergence analysis of a randomized coordinate descent method for large-scale linearly constrained convex optimization, extending Nesterov's techniques to enhance theoretical guarantees.

Contribution

The authors develop new analytical techniques that significantly strengthen convergence results for randomized coordinate descent in linearly constrained problems.

Findings

Enhanced convergence bounds for the algorithm

Extension of Nesterov's second technique to constrained problems

Improved theoretical understanding of the method's efficiency

Abstract

The problem of minimizing a separable convex function under linearly coupled constraints arises from various application domains such as economic systems, distributed control, and network flow. The main challenge for solving this problem is that the size of data is very large, which makes usual gradient-based methods infeasible. Recently, Necoara, Nesterov, and Glineur [Journal of Optimization Theory and Applications, 173 (2017) 227-2254] proposed an efficient randomized coordinate descent method to solve this type of optimization problems and presented an appealing convergence analysis. In this paper, we develop new techniques to analyze the convergence of such algorithms, which are able to greatly improve the results presented there. This refined result is achieved by extending Nesterov's second technique developed by Nesterov [SIAM J. Optim. 22 (2012) 341-362] to the general optimization problems with linearly coupled constraints. A novel technique in our analysis is to establish the basis vectors for the subspace of the linearly constraints.