Murphy Diagrams: Forecast Evaluation of Expected Shortfall

TL;DR

This paper introduces Murphy diagrams, a graphical tool and hypothesis test for evaluating and comparing forecast methods for Expected Shortfall, aiding practitioners in selecting appropriate scoring functions.

Contribution

It develops Murphy diagrams and a hypothesis test to visually and statistically compare forecast methods for Expected Shortfall, addressing practical guidance gaps.

Findings

Murphy diagrams effectively distinguish forecast performance.

The proposed test identifies dominant forecast methods.

Application to stock returns demonstrates practical utility.

Abstract

Motivated by the Basel 3 regulations, recent studies have considered joint forecasts of Value-at-Risk and Expected Shortfall. A large family of scoring functions can be used to evaluate forecast performance in this context. However, little intuitive or empirical guidance is currently available, which renders the choice of scoring function awkward in practice. We therefore develop graphical checks (Murphy diagrams) of whether one forecast method dominates another under a relevant class of scoring functions, and propose an associated hypothesis test. We illustrate these tools with simulation examples and an empirical analysis of S&P 500 and DAX returns.