Extrapolation in Statistical Learning with Extreme Value Theory

TL;DR

This paper reviews how extreme value theory enhances statistical learning for extrapolation tasks in machine learning, especially with limited tail data, covering recent theoretical and practical advances.

Contribution

It synthesizes recent developments at the intersection of statistical learning and extreme value theory, focusing on principled, asymptotically motivated methods for tail extrapolation.

Findings

Provides a comprehensive overview of methods for tail extrapolation in machine learning.

Discusses theoretical frameworks for dependent and independent data in extreme value analysis.

Highlights promising future research directions in the field.

Abstract

Extreme value theory provides rigorous theory and statistical tools for extrapolation in machine learning, particularly in settings where traditional methods struggle due to data scarcity in the tails. A broad range of tasks benefit from these advances, including regression and classification beyond the training data, extreme quantile regression, supervised and unsupervised dimension reduction, generative artificial intelligence and anomaly detection. This review synthesizes recent developments in these fields at the intersection of statistical learning and extreme value theory, with a focus on principled methods based on asymptotically motivated representations of the tail of univariate and multivariate distributions. We consider different theoretical frameworks for both asymptotically dependent and independent data and discuss how they translate into efficient statistical methods for extrapolation to extreme regions. By addressing both theoretical and practical aspects, we offer a comprehensive overview of the state-of-the-art in this quickly evolving field, and identify promising directions for future research.