Backtesting Systemic Risk Forecasts using Multi-Objective Elicitability

TL;DR

This paper introduces a new approach called multi-objective elicitability to enable backtesting of systemic risk measures like CoVaR, CoES, and MES, which were previously impossible to validate due to their non-elicitability.

Contribution

The paper proposes the concept of multi-objective elicitability and develops Diebold--Mariano type tests with two-dimensional scores for systemic risk measures.

Findings

Traffic-light approach for test decisions

Application to DAX 30 and S&P 500 returns

Regulatory recommendations derived

Abstract

Systemic risk measures such as CoVaR, CoES and MES are widely-used in finance, macroeconomics and by regulatory bodies. Despite their importance, we show that they fail to be elicitable and identifiable. This renders forecast comparison and validation, commonly summarised as `backtesting', impossible. The novel notion of \emph{multi-objective elicitability} solves this problem. Specifically, we propose Diebold--Mariano type tests utilising two-dimensional scores equipped with the lexicographic order. We illustrate the test decisions by an easy-to-apply traffic-light approach. We apply our traffic-light approach to DAX~30 and S\&P~500 returns, and infer some recommendations for regulators.