Spurious Predictability in Financial Machine Learning

TL;DR

This paper investigates spurious predictability in financial machine learning, introducing a falsification audit to distinguish genuine signals from artifacts, validated through simulations and empirical case studies.

Contribution

It presents a novel falsification framework that tests predictive workflows against synthetic environments to identify false signals in financial data.

Findings

Many apparent predictability findings are methodological artifacts.

The falsification audit effectively detects spurious signals in simulated environments.

Empirical case studies show that many supposed signals are not genuine.

Abstract

Adaptive specification search generates statistically significant backtests even under martingale-difference nulls. We introduce a falsification audit testing complete predictive workflows against synthetic reference classes, including zero-predictability environments and microstructure placebos. Workflows generating significant walk-forward evidence in these environments are falsified. For passing workflows, we quantify selection-induced performance inflation using an absolute magnitude gap linking optimized in-sample evidence to disjoint walk-forward realizations, adjusted for effective multiplicity. Simulations validate extreme-value scaling under correlated searches and demonstrate detection power under genuine structure. Empirical case studies confirm that many apparent findings represent methodological artifacts rather than genuine predictability.