Evaluation of binary classifiers for asymptotically dependent and independent extremes

TL;DR

This paper develops a risk function tailored for evaluating classifiers that predict rare extreme events, providing a framework grounded in multivariate regular variation, with applications to river discharge forecasting.

Contribution

It introduces a novel risk function for assessing extremal classifiers and analyzes its properties within the multivariate regular variation framework.

Findings

Simulation study compares classifier performance using the new risk function.

Application to Danube river data demonstrates the framework's practical utility.

Abstract

Machine learning classification methods usually assume that all possible classes are sufficiently present within the training set. Due to their inherent rarities, extreme events are always under-represented and classifiers tailored for predicting extremes need to be carefully designed to handle this under-representation. In this paper, we address the question of how to assess and compare classifiers with respect to their capacity to capture extreme occurrences. This is also related to the topic of scoring rules used in forecasting literature. In this context, we propose and study a risk function adapted to extremal classifiers. The inferential properties of our empirical risk estimator are derived under the framework of multivariate regular variation and hidden regular variation. A simulation study compares different classifiers and indicates their performance with respect to our risk function. To conclude, we apply our framework to the analysis of extreme river discharges in the Danube river basin. The application compares different predictive algorithms and test their capacity at forecasting river discharges from other river stations.