Block-proximal methods with spatially adapted acceleration

TL;DR

This paper introduces stochastic primal-dual block-coordinate descent methods with adaptive acceleration and step lengths, achieving optimal convergence rates for convex problems, and demonstrates their effectiveness in image processing tasks.

Contribution

The paper develops doubly-stochastic primal-dual methods with spatially adaptive acceleration and step sizes, extending existing algorithms with new convergence guarantees and practical applicability.

Findings

Achieves $O(1/N^2)$ convergence for strongly convex blocks.

Demonstrates effectiveness in image processing applications.

Introduces blockwise-adapted acceleration and step lengths.

Abstract

We study and develop (stochastic) primal--dual block-coordinate descent methods for convex problems based on the method due to Chambolle and Pock. Our methods have known convergence rates for the iterates and the ergodic gap: $O(1/N^2)$ if each block is strongly convex, $O(1/N)$ if no convexity is present, and more generally a mixed rate $O(1/N^2)+O(1/N)$ for strongly convex blocks, if only some blocks are strongly convex. Additional novelties of our methods include blockwise-adapted step lengths and acceleration, as well as the ability to update both the primal and dual variables randomly in blocks under a very light compatibility condition. In other words, these variants of our methods are doubly-stochastic. We test the proposed methods on various image processing problems, where we employ pixelwise-adapted acceleration.