On the rate of convergence of alternating minimization for non-smooth non-strongly convex optimization in Banach spaces

TL;DR

This paper establishes convergence rates for alternating minimization in non-smooth convex optimization within Banach spaces, showing benefits from block properties and extending previous results to more general settings.

Contribution

It provides new convergence rates for alternating minimization in Banach spaces, considering various relaxations of strong convexity, and highlights the influence of block conditioning on performance.

Findings

Linear convergence for quasi-strong convexity and quadratic growth cases.

Sublinear convergence for plain convexity case.

Performance depends on both best and worst conditioned blocks.

Abstract

In this paper, the convergence of alternating minimization is established for non-smooth convex optimization in Banach spaces, and novel rates of convergence are provided. As objective function a composition of a smooth and a non-smooth part is considered with the latter being block-separable, e.g., corresponding to convex constraints or regularization. For the smooth part, three different relaxations of strong convexity are considered: (i) quasi-strong convexity; (ii) quadratic functional growth; and (iii) plain convexity. Linear convergence is established for the first two cases, generalizing and improving previous results for strongly convex problems; sublinear convergence is established for the third case, also improving previous results from the literature. All the convergence results have in common, that opposing to previous corresponding results for the general block coordinate descent, the performance of the alternating minimization is beneficially governed by properties of the single blocks, instead of global properties. Ultimately, not only the better conditioned block determines the performance, as has been similarly observed in the literature. But also the worse conditioned problem enhances the performance additionally, resulting in potentially significantly improved convergence rates. Furthermore, by solely using the convexity and smoothness properties of the problem, the results immediately apply in general Banach spaces.