A flexible block-coordinate forward-backward algorithm for non-smooth and non-convex optimization

TL;DR

This paper introduces a flexible block-coordinate forward-backward algorithm that guarantees convergence for non-smooth, non-convex optimization, enabling prioritized and correlated block updates, with applications in multilevel image restoration.

Contribution

It proposes a new deterministic BCD framework allowing flexible, prioritized, and correlated block updates while maintaining convergence guarantees for challenging optimization problems.

Findings

Effective in multilevel image restoration tasks

Supports prioritized block updates and correlations

Achieves state-of-the-art convergence guarantees

Abstract

Block coordinate descent (BCD) methods are prevalent in large scale optimization problems due to the low memory and computational costs per iteration, the predisposition to parallelization, and the ability to exploit the structure of the problem. The theoretical and practical performance of BCD relies heavily on the rules defining the choice of the blocks to be updated at each iteration. We propose a new deterministic BCD framework that allows for very flexible updates, while guaranteeing state-of-the-art convergence guarantees on non-smooth nonconvex optimization problems. While encompassing several update rules from the literature, this framework allows for priority on updates of particular blocks and correlations in the block selection between iterations, which is not permitted under the classical convergent stochastic framework. This flexibility is leveraged in the context of multilevel optimization algorithms and, in particular, in multilevel image restoration problems, where the efficiency of the approach is illustrated.