Proper scoring rules for evaluating asymmetry in density forecasting

TL;DR

This paper introduces a new asymmetric scoring rule for evaluating density forecasts, allowing emphasis on specific regions like tails or center, and provides a statistical test for comparing forecast performance, applicable in economic and commodity markets.

Contribution

It develops the asymmetric continuous probabilistic score (ACPS) and a weighted version, enabling tailored evaluation of forecasts based on asymmetric preferences, with applications to macroeconomic and commodity data.

Findings

ACPS effectively emphasizes regions of interest in forecast evaluation.

The score and test distinguish forecast performance during COVID-19 crisis.

Application to real datasets demonstrates practical usefulness.

Abstract

This paper proposes a novel asymmetric continuous probabilistic score (ACPS) for evaluating and comparing density forecasts. It extends the proposed score and defines a weighted version, which emphasizes regions of interest, such as the tails or the center of a variable's range. A test is also introduced to statistically compare the predictive ability of different forecasts. The ACPS is of general use in any situation where the decision maker has asymmetric preferences in the evaluation of the forecasts. In an artificial experiment, the implications of varying the level of asymmetry in the ACPS are illustrated. Then, the proposed score and test are applied to assess and compare density forecasts of macroeconomic relevant datasets (US employment growth) and of commodity prices (oil and electricity prices) with particular focus on the recent COVID-19 crisis period.