An inexact restoration-nonsmooth algorithm with variable accuracy for stochastic nonsmooth convex optimization problems in machine learning and stochastic linear complementarity problems

TL;DR

This paper introduces an inexact restoration-nonsmooth algorithm with adaptive sample sizes for stochastic convex optimization, demonstrating convergence and efficiency in machine learning and complementarity problems.

Contribution

It presents a novel adaptive sampling method within an inexact restoration framework for stochastic nonsmooth convex optimization, with proven convergence.

Findings

Algorithm converges almost surely under standard assumptions.

Numerical results show efficiency in machine learning and complementarity problems.

Adaptive sample size selection improves computational performance.

Abstract

We study unconstrained optimization problems with nonsmooth and convex objective function in the form of a mathematical expectation. The proposed method approximates the expected objective function with a sample average function using Inexact Restoration-based adapted sample sizes. The sample size is chosen in an adaptive manner based on Inexact Restoration. The algorithm uses line search and assumes descent directions with respect to the current approximate function. We prove the a.s. convergence under standard assumptions. Numerical results for two types of problems, machine learning loss function for training classifiers and stochastic linear complementarity problems, prove the efficiency of the proposed scheme.