Empirical risk minimization for heavy-tailed losses

TL;DR

This paper explores empirical risk minimization with heavy-tailed losses, proposing the use of Catoni's robust mean estimator to improve reliability and derive performance bounds.

Contribution

It introduces a novel analysis of empirical risk minimization using Catoni's estimator for heavy-tailed losses, with tailored performance bounds.

Findings

Robust risk minimization is feasible with heavy-tailed data.

Performance bounds are established using chaining arguments.

Catoni's estimator improves estimation reliability in heavy-tailed scenarios.

Abstract

The purpose of this paper is to discuss empirical risk minimization when the losses are not necessarily bounded and may have a distribution with heavy tails. In such situations, usual empirical averages may fail to provide reliable estimates and empirical risk minimization may provide large excess risk. However, some robust mean estimators proposed in the literature may be used to replace empirical means. In this paper, we investigate empirical risk minimization based on a robust estimate proposed by Catoni. We develop performance bounds based on chaining arguments tailored to Catoni's mean estimator.