Convergence rate analysis of randomized and cyclic coordinate descent for convex optimization through semidefinite programming

TL;DR

This paper analyzes the convergence rates of randomized and cyclic coordinate descent methods for convex optimization, using semidefinite programming to improve known bounds and providing insights into the Gauss-Seidel method for positive semidefinite systems.

Contribution

It introduces a semidefinite programming approach to better estimate convergence rates of coordinate descent methods and applies it to analyze the Gauss-Seidel method's worst-case performance.

Findings

Improved convergence rate bounds for coordinate descent methods.

New semidefinite programming technique for performance estimation.

Analysis of Gauss-Seidel method for positive semidefinite matrices.

Abstract

In this paper, we study randomized and cyclic coordinate descent for convex unconstrained optimization problems. We improve the known convergence rates in some cases by using the numerical semidefinite programming performance estimation method. As a spin-off we provide a method to analyse the worst-case performance of the Gauss-Seidel iterative method for linear systems where the coefficient matrix is positive semidefinite with a positive diagonal.