Stochastic Optimization for Spectral Risk Measures

TL;DR

This paper introduces stochastic algorithms for optimizing spectral risk measures, which balance average and worst-case performance, overcoming biases and non-smoothness issues present in traditional methods.

Contribution

The paper develops novel stochastic optimization algorithms specifically designed for spectral risk measures, addressing bias and non-smoothness challenges.

Findings

Our algorithms outperform standard stochastic subgradient methods.

Theoretical analysis confirms convergence properties.

Experimental results demonstrate improved optimization of spectral risk objectives.

Abstract

Spectral risk objectives - also called $L$-risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop stochastic algorithms to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging are hindered by bias and that our approach outperforms them.