Optimizing Shortfall Risk Metric for Learning Regression Models

TL;DR

This paper introduces a method to estimate and optimize the utility-based shortfall risk (UBSR) in regression, providing theoretical bounds and an algorithm for finding the UBSR-optimal model.

Contribution

It develops a concentration bound for UBSR estimation, formulates UBSR optimization as a pseudo-linear problem, and proposes a convergent algorithm using gradient and linear minimization oracles.

Findings

Derived a concentration bound for UBSR estimation.

Formulated UBSR optimization as a pseudo-linear problem.

Designed a convergent bisection-type algorithm for UBSR optimization.

Abstract

We consider the problem of estimating and optimizing utility-based shortfall risk (UBSR) of a loss, say $(Y - \hat Y)^2$, in the context of a regression problem. Empirical risk minimization with a UBSR objective is challenging since UBSR is a non-linear function of the underlying distribution. We first derive a concentration bound for UBSR estimation using independent and identically distributed (i.i.d.) samples. We then frame the UBSR optimization problem as minimization of a pseudo-linear function in the space of achievable distributions $\mathcal D$ of the loss $(Y- \hat Y)^2$. We construct a gradient oracle for the UBSR objective and a linear minimization oracle (LMO) for the set $\mathcal D$. Using these oracles, we devise a bisection-type algorithm, and establish convergence to the UBSR-optimal solution.