Algorithms based on operator-averaged operators

TL;DR

This paper introduces the concept of operator-averaged operators to analyze and develop fast algorithms for convex minimization problems with sparsity, demonstrating improved convergence over traditional methods through theoretical insights and experiments.

Contribution

It extends the framework of scalar-averaged operators to operator-averaged operators, enabling analysis and creation of more efficient algorithms for sparse convex optimization.

Findings

Operator-averaged operators facilitate the study of certain fast algorithms.

Algorithms based on operator-averaged operators converge faster than conventional methods.

Experimental results confirm the efficiency of the proposed algorithms.

Abstract

A class of algorithms comprised by certain semismooth Newton and active-set methods is able to solve convex minimization problems involving sparsity-inducing regularizers very rapidly; the speed advantage of methods from this class is a consequence of their ability to benefit from the sparsity of the corresponding solutions by solving smaller inner problems than conventional methods. The convergence properties of such conventional methods (e.g., the forward-backward and the proximal-Newton ones) can be studied very elegantly under the framework of iterations of scalar-averaged operators - this is not the case for the aforementioned class. However, we show in this work that by instead considering operator-averaged operators, one can indeed study methods of that class, and also to derive algorithms outside of it that may be more convenient to implement than existing ones. Additionally, we present experiments whose results suggest that methods based on operator-averaged operators achieve substantially faster convergence than conventional ones.