Variants of alternating minimization method with sublinear rates of convergence for convex optimization

TL;DR

This paper introduces two new variants of the alternating minimization method for convex optimization, achieving sublinear convergence rates and suitable for difficult subproblems and parallel computation.

Contribution

The paper proposes novel proximal linearized variants of AM with proven sublinear convergence under minimal assumptions, enhancing efficiency for complex and parallelizable problems.

Findings

Both variants achieve sublinear convergence rates.

The first variant handles hard subproblems efficiently.

The second variant is optimized for parallel computation.

Abstract

The alternating minimization (AM) method is a fundamental method for minimizing convex functions whose variable consists of two blocks. How to efficiently solve each subproblems when applying the AM method is the most concerned task. In this paper, we investigate this task and design two new variants of the AM method by borrowing proximal linearized techniques. The first variant is very suitable for the case where half of the subproblems are hard to be solved and the other half can be directly computed. The second variant is designed for parallel computation. Both of them are featured by simplicity at each iteration step. Theoretically, with the help of the proximal operator we first write the new as well as the existing AM variants into uniform expressions, and then prove that they enjoy sublinear rates of convergence under very minimal assumptions.