Randomized Block Coordinate DC Programming

TL;DR

This paper extends the Difference of Convex Algorithm to a randomized block coordinate method with proven convergence rates, and introduces a related randomized block coordinate EM algorithm for problems with separable structure.

Contribution

It presents a novel randomized block coordinate extension of DCA with convergence guarantees and connects it to EM, offering new algorithms for structured non-convex problems.

Findings

Non-asymptotic convergence rate of O(n/k) in expectation.

Introduction of a randomized block coordinate EM algorithm.

Theoretical analysis of convergence for the proposed methods.

Abstract

We introduce an extension of the Difference of Convex Algorithm (DCA) in the form of a randomized block coordinate approach for problems with separable structure. For $n$ coordinate-blocks and $k$ iterations, our main result proves a non-asymptotic convergence rate of $O(n/k)$ in expectation, with respect to a stationarity measure based on a Forward-Backward envelope. Furthermore, leveraging the connection between DCA and Expectation Maximization (EM), we propose a randomized block coordinate EM algorithm.