A Field Guide to Forward-Backward Splitting with a FASTA Implementation

TL;DR

This paper reviews the practical aspects of forward-backward splitting algorithms, emphasizing implementation details, and introduces FASTA, a versatile solver that incorporates various algorithmic variations for high-dimensional optimization problems.

Contribution

It provides a comprehensive review of forward-backward splitting with practical guidance and introduces FASTA, a flexible implementation that includes multiple algorithmic variations.

Findings

FASTA effectively implements various forward-backward splitting variants.

Implementation details like stepsize and acceleration significantly impact performance.

Numerical experiments demonstrate the effectiveness of different approaches.

Abstract

Non-differentiable and constrained optimization play a key role in machine learning, signal and image processing, communications, and beyond. For high-dimensional minimization problems involving large datasets or many unknowns, the forward-backward splitting method provides a simple, practical solver. Despite its apparently simplicity, the performance of the forward-backward splitting is highly sensitive to implementation details. This article is an introductory review of forward-backward splitting with a special emphasis on practical implementation concerns. Issues like stepsize selection, acceleration, stopping conditions, and initialization are considered. Numerical experiments are used to compare the effectiveness of different approaches. Many variations of forward-backward splitting are implemented in the solver FASTA (short for Fast Adaptive Shrinkage/Thresholding Algorithm). FASTA provides a simple interface for applying forward-backward splitting to a broad range of problems.