Estimating real-world probabilities: A forward-looking behavioral framework

TL;DR

This paper introduces a behavioral framework that improves real-world probability forecasts by disentangling sentiment biases from fundamental expectations, demonstrating consistent gains across various models and data.

Contribution

It presents a novel behavioral transformation method that enhances probabilistic forecasting accuracy and robustness in financial markets.

Findings

Behavioral transformation improves forecast accuracy

Robust across different models and sentiment calibrations

Outperforms recalibrated densities and enhances risk estimation

Abstract

We show that disentangling sentiment-induced biases from fundamental expectations significantly improves the accuracy and consistency of probabilistic forecasts. Using data from 1994 to 2017, we analyze 15 stochastic models and risk-preference combinations and in all possible cases a simple behavioral transformation delivers substantial forecast gains. Our results are robust across different evaluation methods, risk-preference hypotheses and sentiment calibrations, demonstrating that behavioral effects can be effectively used to forecast asset prices. Further analyses confirm that our real-world densities outperform densities recalibrated to avoid past mistakes and improve predictive models where risk aversion is dynamically estimated from option prices.