Sharper Bounds for Proximal Gradient Algorithms with Errors

TL;DR

This paper derives tighter convergence bounds for proximal gradient algorithms with errors, demonstrating improved robustness and accuracy in reduced-precision and inexact computations for convex optimization.

Contribution

It introduces new deterministic and probabilistic bounds for inexact proximal gradient methods, including analysis of error amplification due to acceleration.

Findings

Probabilistic bounds are more robust for verification.

Acceleration amplifies computational errors.

Bounds are validated on MPC and LASSO problems.

Abstract

We analyse the convergence of the proximal gradient algorithm for convex composite problems in the presence of gradient and proximal computational inaccuracies. We derive new tighter deterministic and probabilistic bounds that we use to verify a simulated (MPC) and a synthetic (LASSO) optimization problems solved on a reduced-precision machine in combination with an inaccurate proximal operator. We also show how the probabilistic bounds are more robust for algorithm verification and more accurate for application performance guarantees. Under some statistical assumptions, we also prove that some cumulative error terms follow a martingale property. And conforming to observations, e.g., in \cite{schmidt2011convergence}, we also show how the acceleration of the algorithm amplifies the gradient and proximal computational errors.