An Accelerated Variance Reduced Extra-Point Approach to Finite-Sum VI and Optimization

TL;DR

This paper introduces accelerated variance reduced algorithms for finite-sum hemivariational inequalities and optimization problems, achieving optimal gradient complexities and demonstrating effectiveness through numerical experiments.

Contribution

The paper develops novel variance reduction algorithms for finite-sum HVIs with accelerated convergence matching best known bounds.

Findings

Achieved optimal gradient complexity bounds for the proposed algorithms.

Demonstrated the algorithms' effectiveness on constrained finite-sum optimization problems.

Provided preliminary numerical results supporting the theoretical findings.

Abstract

In this paper, we develop stochastic variance reduced algorithms for solving a class of finite-sum hemivariational inequality (HVI) problem. In this HVI problem, the associated function is assumed to be differentiable, and both the vector mapping and the function are of finite-sum structure. We propose two algorithms to solve the cases when the vector mapping is either merely monotone or strongly monotone, while the function is assumed to be convex. We show how to apply variance reduction in the proposed algorithms when such an HVI problem has a finite-sum structure, and the resulting accelerated gradient complexities can match the best bound established for finite-sum VI problem, as well as the bound given by the direct Katyusha for finite-sum optimization respectively, in terms of the corresponding parameters such as (gradient) Lipschitz constants and the sizes of the finite-sums. We demonstrate the application of our algorithms through solving a finite-sum constrained finite-sum optimization problem and provide preliminary numerical results.