Superquantiles at Work: Machine Learning Applications and Efficient Subgradient Computation

TL;DR

This paper reviews recent machine learning applications of superquantiles, focusing on their nonsmooth nature, smoothing techniques via infimal convolution, and efficient subgradient computation for gradient-based algorithms.

Contribution

It introduces a smoothing approach for superquantile-based functions and provides numerical procedures for gradient computation, enhancing their applicability in machine learning.

Findings

Superquantile functions can be smoothed using infimal convolution.

Explicit subgradients facilitate optimization of nonsmooth superquantile objectives.

Numerical procedures enable efficient gradient computation for superquantile-based models.

Abstract

R. Tyrell Rockafellar and collaborators introduced, in a series of works, new regression modeling methods based on the notion of superquantile (or conditional value-at-risk). These methods have been influential in economics, finance, management science, and operations research in general. Recently, they have been the subject of a renewed interest in machine learning, to address issues of distributional robustness and fair allocation. In this paper, we review some of these new applications of the superquantile, with references to recent developments. These applications involve nonsmooth superquantile-based objective functions that admit explicit subgradient calculations. To make these superquantile-based functions amenable to the gradient-based algorithms popular in machine learning, we show how to smooth them by infimal convolution and describe numerical procedures to compute the gradients of the smooth approximations. We put the approach into perspective by comparing it to other smoothing techniques and by illustrating it on toy examples.