Proximal Flow Inspired Multi-Step Methods

TL;DR

This paper introduces a family of multi-step proximal point methods inspired by gradient flow discretizations, demonstrating improved convergence in various optimization settings with comparable computational cost to traditional methods.

Contribution

It proposes a novel multi-step proximal point framework, providing convergence analysis across diverse problem types and showing accelerated results for specific cases.

Findings

Improved convergence behavior in multiple optimization scenarios.

Convergence guarantees for quadratic, strongly/weakly convex, and nonconvex problems.

Accelerated convergence for alternating projections.

Abstract

We investigate a family of approximate multi-step proximal point methods, framed as implicit linear discretizations of gradient flow. The resulting methods are multi-step proximal point methods, with similar computational cost in each update as the proximal point method. We explore several optimization methods where applying an approximate multistep proximal points method results in improved convergence behavior. We also include convergence analysis for the proposed method in several problem settings: quadratic problems, general problems that are strongly or weakly (non)convex, and accelerated results for alternating projections.