Complexity of Inexact Proximal Point Algorithm for minimizing convex functions with Holderian Growth

TL;DR

This paper analyzes the iteration complexity of inexact proximal point algorithms for convex functions with Holderian growth, providing new bounds and demonstrating improvements over existing methods through numerical tests.

Contribution

It derives nonasymptotic complexity bounds for inexact PPA under Holderian growth and introduces novel computational bounds for restarting inexact PPA using a proximal subgradient method.

Findings

Nonasymptotic iteration complexity bounds for exact and inexact PPA.

Recovery of classical convergence results like finite and linear convergence.

Numerical improvements over existing restarting subgradient methods.

Abstract

Several decades ago the Proximal Point Algorithm (PPA) started to gain a long-lasting attraction for both abstract operator theory and numerical optimization communities. Even in modern applications, researchers still use proximal minimization theory to design scalable algorithms that overcome nonsmoothness. Remarkable works as \cite{Fer:91,Ber:82constrained,Ber:89parallel,Tom:11} established tight relations between the convergence behaviour of PPA and the regularity of the objective function. In this manuscript we derive nonasymptotic iteration complexity of exact and inexact PPA to minimize convex functions under $\gamma-$Holderian growth: $\BigO{\log(1/\epsilon)}$ (for $\gamma \in [1,2]$) and $\BigO{1/\epsilon^{\gamma - 2}}$ (for $\gamma > 2$). In particular, we recover well-known results on PPA: finite convergence for sharp minima and linear convergence for quadratic growth, even under presence of deterministic noise. Moreover, when a simple Proximal Subgradient Method is recurrently called as an inner routine for computing each IPPA iterate, novel computational complexity bounds are obtained for Restarting Inexact PPA. Our numerical tests show improvements over existing restarting versions of the Subgradient Method.