On Monte-Carlo methods in convex stochastic optimization

TL;DR

This paper introduces a median-of-means tournament method for convex stochastic optimization that achieves optimal statistical performance in heavy-tailed data, applicable in high-dimensional settings with explicit sample size thresholds.

Contribution

It presents the first non-asymptotic, optimal-rate procedure for convex stochastic optimization under heavy tails, extending to high-dimensional problems with explicit sample size dependence.

Findings

Optimal non-asymptotic rates in heavy-tailed stochastic optimization

Applicability to high-dimensional problems with explicit sample size thresholds

Recovery of recent results in multivariate mean estimation and linear regression

Abstract

We develop a novel procedure for estimating the optimizer of general convex stochastic optimization problems of the form $\min_{x\in\mathcal{X}} \mathbb{E}[F(x,\xi)]$, when the given data is a finite independent sample selected according to $\xi$. The procedure is based on a median-of-means tournament, and is the first procedure that exhibits the optimal statistical performance in heavy tailed situations: we recover the asymptotic rates dictated by the central limit theorem in a non-asymptotic manner once the sample size exceeds some explicitly computable threshold. Additionally, our results apply in the high-dimensional setup, as the threshold sample size exhibits the optimal dependence on the dimension (up to a logarithmic factor). The general setting allows us to recover recent results on multivariate mean estimation and linear regression in heavy-tailed situations and to prove the first sharp, non-asymptotic results for the portfolio optimization problem.