Risk minimization by median-of-means tournaments

TL;DR

This paper introduces a median-of-means tournament method for regression that optimally balances accuracy and confidence, outperforming traditional empirical risk minimization under minimal assumptions.

Contribution

The paper proposes a novel median-of-means tournament procedure that achieves optimal accuracy-confidence tradeoff in regression tasks, improving upon classical methods.

Findings

Outperforms empirical risk minimization in regression.

Achieves optimal tradeoff between accuracy and confidence.

Works under minimal assumptions.

Abstract

We consider the classical statistical learning/regression problem, when the value of a real random variable Y is to be predicted based on the observation of another random variable X. Given a class of functions F and a sample of independent copies of (X, Y ), one needs to choose a function f from F such that f(X) approximates Y as well as possible, in the mean-squared sense. We introduce a new procedure, the so-called median-of-means tournament, that achieves the optimal tradeoff between accuracy and confidence under minimal assumptions, and in particular outperforms classical methods based on empirical risk minimization.