Lyapunov Analysis For Monotonically Forward-Backward Accelerated Algorithms

TL;DR

This paper provides a Lyapunov-based analysis of monotonic variants of Nesterov's accelerated methods, establishing their linear convergence under strong convexity without relying on full NAG iterates.

Contribution

It introduces a Lyapunov function approach for M-NAG and M-FISTA, proving their linear convergence and clarifying the role of auxiliary sequences and assumptions.

Findings

Lyapunov function guarantees linear convergence of M-NAG.

Modified sequences enable direct extension to M-FISTA.

Convergence results depend only on the momentum parameter, not problem constants.

Abstract

Nesterov's accelerated gradient method (NAG) achieves faster convergence than gradient descent for convex optimization but lacks monotonicity in function values. To address this, Beck and Teboulle [2009b] proposed a monotonic variant, M-NAG, and extended it to the proximal setting as M-FISTA for composite problems such as Lasso. However, establishing the linear convergence of M-NAG and M-FISTA under strong convexity remains an open problem. In this paper, we analyze M-NAG via the implicit-velocity phase representation and show that an additional assumption, either the position update or the phase-coupling relation, is necessary to fully recover the NAG iterates. The essence of M-NAG lies in controlling an auxiliary sequence to enforce non-increase. We further demonstrate that the M-NAG update alone is sufficient to construct a Lyapunov function guaranteeing linear convergence, without relying on full NAG iterates. By modifying the mixed sequence to incorporate forward-indexed gradients, we develop a new Lyapunov function that removes the kinetic energy term, enabling a direct extension to M-NAG. The required starting index depends only on the momentum parameter and not on problem constants. Finally, leveraging newly developed proximal inequalities, we extend our results to M-FISTA, establishing its linear convergence and deepening the theoretical understanding of monotonic accelerated methods.