Learning with risks based on M-location

TL;DR

This paper introduces a new class of risk measures based on M-location, extending traditional risk functions, with practical implementation, theoretical guarantees, interpretability, and impact on test loss distribution.

Contribution

It proposes a novel risk framework based on M-location that generalizes classical risk functions, offering theoretical guarantees and practical advantages.

Findings

Provides finite-sample stationarity guarantees for stochastic gradient methods.

Easily implemented as a wrapper around smooth loss functions.

Significantly influences the test loss distribution.

Abstract

In this work, we study a new class of risks defined in terms of the location and deviation of the loss distribution, generalizing far beyond classical mean-variance risk functions. The class is easily implemented as a wrapper around any smooth loss, it admits finite-sample stationarity guarantees for stochastic gradient methods, it is straightforward to interpret and adjust, with close links to M-estimators of the loss location, and has a salient effect on the test loss distribution.