On the Proximal Gradient Algorithm with Alternated Inertia

TL;DR

This paper studies a variant of the proximal gradient algorithm that uses alternated inertia, demonstrating its monotonic convergence and providing convergence rates based on local geometry, with practical illustrations.

Contribution

It introduces and analyzes the proximal gradient algorithm with alternated inertia, showing its advantages over traditional accelerated methods.

Findings

Monotonically decreasing functional values with alternated inertia

Convergence rates based on local geometric properties

Effective in common regularized problems

Abstract

In this paper, we investigate the attractive properties of the proximal gradient algorithm with inertia. Notably, we show that using alternated inertia yields monotonically decreasing functional values, which contrasts with usual accelerated proximal gradient methods. We also provide convergence rates for the algorithm with alternated inertia based on local geometric properties of the objective function. The results are put into perspective by discussions on several extensions and illustrations on common regularized problems.