Anderson Accelerated Operator Splitting Methods for Convex-nonconvex Regularized Problems

TL;DR

This paper introduces Anderson accelerated operator splitting methods for convex-nonconvex regularized problems, proving their convergence and demonstrating practical speed-ups in applications like signal processing and machine learning.

Contribution

It develops and analyzes Anderson accelerated operator splitting algorithms specifically for CNC regularized problems, a novel approach in this context.

Findings

Proven global convergence of the proposed algorithms.

Achieved significant practical speed-ups in applications.

Applicable to signal processing, statistics, and machine learning.

Abstract

Convex-nonconvex (CNC) regularization is a novel paradigm that employs a nonconvex penalty function while maintaining the convexity of the entire objective function. It has been successfully applied to problems in signal processing, statistics, and machine learning. Despite its wide application, the computation of CNC regularized problems remains challenging and under-investigated. To fill the gap, we study several operator splitting methods and their Anderson accelerated counterparts for solving least squares problems with CNC regularization. We establish the global convergence of the proposed algorithm to an optimal point and demonstrate its practical speed-ups in various applications.