Spectral backtests of forecast distributions with application to risk management

TL;DR

This paper introduces spectral backtests for forecast distributions that incorporate user-defined weighting schemes, enabling more flexible and powerful assessments of risk models, demonstrated through empirical analysis of bank trading portfolios.

Contribution

It proposes a new class of spectral backtests that unify and extend existing methods, with novel variants that are easy to implement and have strong statistical properties.

Findings

Test results vary with different kernel choices.

Spectral backtests effectively evaluate risk forecast distributions.

Application to bank portfolios shows practical relevance.

Abstract

We study a class of backtests for forecast distributions in which the test statistic depends on a spectral transformation that weights exceedance events by a function of the modeled probability level. The weighting scheme is specified by a kernel measure which makes explicit the user's priorities for model performance. The class of spectral backtests includes tests of unconditional coverage and tests of conditional coverage. We show how the class embeds a wide variety of backtests in the existing literature, and further propose novel variants which are easily implemented, well-sized and have good power. In an empirical application, we backtest forecast distributions for the overnight P&L of ten bank trading portfolios. For some portfolios, test results depend materially on the choice of kernel.