Superquantile-based learning: a direct approach using gradient-based optimization

TL;DR

This paper introduces a new supervised learning approach that uses superquantile risk measures to improve robustness against distributional shifts, optimized efficiently via gradient-based methods.

Contribution

It presents a novel formulation of supervised learning with superquantile risk, enabling direct gradient-based optimization and providing a software implementation for practical use.

Findings

Superquantile-based objective is compatible with gradient optimization methods.

The approach enhances robustness to distributional shifts.

Software SPQR facilitates experimentation with the method.

Abstract

We consider a formulation of supervised learning that endows models with robustness to distributional shifts from training to testing. The formulation hinges upon the superquantile risk measure, also known as the conditional value-at-risk, which has shown promise in recent applications of machine learning and signal processing. We show that, thanks to a direct smoothing of the superquantile function, a superquantile-based learning objective is amenable to gradient-based optimization, using batch optimization algorithms such as gradient descent or quasi-Newton algorithms, or using stochastic optimization algorithms such as stochastic gradient algorithms. A companion software SPQR implements in Python the algorithms described and allows practitioners to experiment with superquantile-based supervised learning.