On the Generalization and Robustness in Conditional Value-at-Risk

TL;DR

This paper provides a comprehensive theoretical analysis of CVaR-based learning under heavy-tailed and contaminated data, establishing bounds, robustness guarantees, and fundamental limitations related to tail behavior.

Contribution

It develops new generalization and robustness bounds for CVaR learning, introduces a robust estimator, and characterizes the intrinsic instability of CVaR decisions in heavy-tailed regimes.

Findings

Established sharp high-probability generalization bounds for CVaR

Proposed a truncated median-of-means CVaR estimator with optimal rates

Identified fundamental instability of CVaR decisions under heavy tails

Abstract

Conditional Value-at-Risk (CVaR) is a widely used risk-sensitive objective for learning under rare but high-impact losses, yet its statistical behavior under heavy-tailed data remains poorly understood. Unlike expectation-based risk, CVaR depends on an endogenous, data-dependent quantile, which couples tail averaging with threshold estimation and fundamentally alters both generalization and robustness properties. In this work, we develop a learning-theoretic analysis of CVaR-based empirical risk minimization under heavy-tailed and contaminated data. We establish sharp, high-probability generalization and excess risk bounds under minimal moment assumptions, covering fixed hypotheses, finite and infinite classes, and extending to $\beta$-mixing dependent data; we further show that these rates are minimax optimal. To capture the intrinsic quantile sensitivity of CVaR, we derive a uniform Bahadur-Kiefer type expansion that isolates a threshold-driven error term absent in mean-risk ERM and essential in heavy-tailed regimes. We complement these results with robustness guarantees by proposing a truncated median-of-means CVaR estimator that achieves optimal rates under adversarial contamination. Finally, we show that CVaR decisions themselves can be intrinsically unstable under heavy tails, establishing a fundamental limitation on decision robustness even when the population optimum is well separated. Together, our results provide a principled characterization of when CVaR learning generalizes and is robust, and when instability is unavoidable due to tail scarcity.