Tile test for back-testing risk evaluation

TL;DR

This paper introduces a novel back-testing method for financial risk models based on the probability integral transform, which effectively evaluates both distributional and dynamic aspects of risk estimations using tiling techniques.

Contribution

It presents a new, powerful test for assessing the accuracy of risk estimations that considers dynamic market behavior and distributional properties, improving upon existing benchmarks.

Findings

The test effectively detects inaccuracies in risk models.

It highlights the importance of capturing market dynamics.

Some widely used risk methodologies are shown to be inadequate.

Abstract

A new test for measuring the accuracy of financial market risk estimations is introduced. It is based on the probability integral transform (PIT) of the ex post realized returns using the ex ante probability distributions underlying the risk estimation. If the forecast is correct, the result of the PIT, that we called probtile, should be an iid random variable with a uniform distribution. The new test measures the variance of the number of probtiles in a tiling over the whole sample. Using different tilings allow to check the dynamic and the distributional aspect of risk methodologies. The new test is very powerful, and new benchmarks need to be introduced to take into account subtle mean reversion effects induced by some risk estimations. The test is applied on 2 data sets for risk horizons of 1 and 10 days. The results show unambiguously the importance of capturing correctly the dynamic of the financial market, and exclude some broadly used risk methodologies.