Regularized quasi-monotone method for stochastic optimization

TL;DR

This paper introduces a stochastic optimization method based on quasi-monotone techniques, achieving optimal convergence rates for the last iterate and validated through experiments on l1-regularized robust linear regression.

Contribution

It adapts the quasi-monotone method for stochastic convex minimization and establishes optimal convergence rates for the last iterate, a novel theoretical advancement.

Findings

Optimal convergence rate in last-iterate for stochastic setting

Confirmed theoretical guarantees with numerical experiments

Effective on l1-regularized robust linear regression

Abstract

We adapt the quasi-monotone method from [2] for composite convex minimization in the stochastic setting. For the proposed numerical scheme we derive the optimal convergence rate in terms of the last iterate, rather than on average as it is standard for subgradient methods. The theoretical guarantee for individual convergence of the regularized quasi-monotone method is confirmed by numerical experiments on l1-regularized robust linear regression.