Follow the flow: Proximal flow inspired multi-step methods

TL;DR

This paper introduces a family of accelerated multi-step proximal point methods inspired by gradient flow discretizations, demonstrating improved convergence through reduced truncation error in approximation.

Contribution

It proposes a novel class of multi-step proximal methods accelerated by implicit discretizations, enhancing convergence efficiency in optimization algorithms.

Findings

Improved convergence behavior in several optimization methods.

Approximate multi-step proximal methods have similar computational cost as standard proximal point methods.

Lowering truncation error leads to better approximation of gradient flow.

Abstract

We investigate a family of approximate multi-step proximal point methods, accelerated by implicit linear discretizations of gradient flow. The resulting methods are multi-step proximal point methods, with similar computational cost in each update as the proximal point method. We explore several optimization methods where applying an approximate multistep proximal points method results in improved convergence behavior. We argue that this is the result of the lowering of truncation error in approximating gradient flow