A Semismooth Newton Stochastic Proximal Point Algorithm with Variance Reduction

TL;DR

This paper introduces a stochastic proximal point algorithm with variance reduction and semismooth Newton updates, providing convergence guarantees and demonstrating superior robustness and performance in numerical experiments.

Contribution

It presents a novel stochastic proximal point method incorporating variance reduction and inexact semismooth Newton updates for weakly convex optimization.

Findings

Algorithm achieves competitive performance with state-of-the-art methods.

Provides detailed convergence analysis accounting for inexact updates.

Demonstrates higher robustness to step size variations.

Abstract

We develop an implementable stochastic proximal point (SPP) method for a class of weakly convex, composite optimization problems. The proposed stochastic proximal point algorithm incorporates a variance reduction mechanism and the resulting SPP updates are solved using an inexact semismooth Newton framework. We establish detailed convergence results that take the inexactness of the SPP steps into account and that are in accordance with existing convergence guarantees of (proximal) stochastic variance-reduced gradient methods. Numerical experiments show that the proposed algorithm competes favorably with other state-of-the-art methods and achieves higher robustness with respect to the step size selection.