Cross-validation on Extreme Regions

TL;DR

This paper provides finite-sample probabilistic bounds for cross-validation estimates of generalization error in extreme regions of the covariate space, addressing challenges posed by small sample sizes and rare events.

Contribution

It introduces new exponential and polynomial probability bounds for CV error in extreme value settings, extending existing guarantees to rare event scenarios.

Findings

Bounds are sharp and match standard CV guarantees.

Results are applicable to high-dimensional classification tasks.

Numerical experiments confirm the bounds' tightness.

Abstract

We conduct a non asymptotic study of the Cross Validation (CV) estimate of the generalization risk for learning algorithms dedicated to extreme regions of the covariates space. In this Extreme Value Analysis context, the risk function measures the algorithm's error given that the norm of the input exceeds a high quantile. The main challenge within this framework is the negligible size of the extreme training sample with respect to the full sample size and the necessity to re-scale the risk function by a probability tending to zero. We open the road to a finite sample understanding of CV for extreme values by establishing two new results: an exponential probability bound on the \Kfold CV error and a polynomial probability bound on the leave-\textrm{p}-out CV. Our bounds are sharp in the sense that they match state-of-the-art guarantees for standard CV estimates while extending them to encompass a conditioning event of small probability. We illustrate the significance of our results regarding high dimensional classification in extreme regions via a Lasso-type logistic regression algorithm. The tightness of our bounds is investigated in numerical experiments.