Validation of credit default probabilities via multiple testing procedures

TL;DR

This paper evaluates multiple testing procedures to validate estimated default probabilities in credit rating systems, aiming to identify inaccuracies while controlling error rates through simulations and empirical data analysis.

Contribution

It introduces methods for validating credit default probabilities that control for type I errors, considering data discreteness and applying to real-world credit data.

Findings

Procedures effectively identify misestimated rating classes.

Methods control familywise error rate and false discovery rate.

Simulation and empirical results demonstrate practical applicability.

Abstract

We apply multiple testing procedures to the validation of estimated default probabilities in credit rating systems. The goal is to identify rating classes for which the probability of default is estimated inaccurately, while still maintaining a predefined level of committing type I errors as measured by the familywise error rate (FWER) and the false discovery rate (FDR). For FWER, we also consider procedures that take possible discreteness of the data resp. test statistics into account. The performance of these methods is illustrated in a simulation setting and for empirical default data.