Conditional Method Confidence Set

TL;DR

This paper introduces a Conditional Method Confidence Set (CMCS) for selecting the best forecasting methods based on economic regimes, enhancing stress-testing of financial risk models with proven asymptotic validity and empirical relevance.

Contribution

It adapts the Model Confidence Set to conditional forecast evaluation, providing a robust tool for regime-specific forecast selection and stress-testing in financial risk management.

Findings

CMCS is asymptotically valid for conditional forecast evaluation.

The method effectively identifies best models across different economic regimes.

Empirical application demonstrates its usefulness in stress-testing market risk models.

Abstract

This paper proposes a Conditional Method Confidence Set (CMCS) which allows to select the best subset of forecasting methods with equal predictive ability conditional on a specific economic regime. The test resembles the Model Confidence Set by Hansen et al. (2011) and is adapted for conditional forecast evaluation. We show the asymptotic validity of the proposed test and illustrate its properties in a simulation study. The proposed testing procedure is particularly suitable for stress-testing of financial risk models required by the regulators. We showcase the empirical relevance of the CMCS using the stress-testing scenario of Expected Shortfall. The empirical evidence suggests that the proposed CMCS procedure can be used as a robust tool for forecast evaluation of market risk models for different economic regimes.