The Oracle's Fingerprint: Correlated AI Forecasting Errors and the Limits of Bias Transmission

TL;DR

This paper investigates how large language models exhibit highly correlated forecasting errors, revealing an emerging 'epistemic monoculture' that amplifies shared biases and limits collective intelligence.

Contribution

It provides empirical evidence of correlated errors among independent LLMs and examines their influence on human forecasts, highlighting the limits of bias transmission.

Findings

LLMs show highly correlated forecasting errors (r ≈ 0.77).

Community forecasts shift predictably but are driven by rational updating.

Human biases pre-ChatGPT already resembled LLM biases; post-ChatGPT resemblance decreased.

Abstract

When large language models (LLMs) are consulted as forecasting tools, the independence of individual errors -- the foundation of collective intelligence -- may collapse. We test three conditions necessary for this "epistemic monoculture" to emerge. In Study 1, we show that GPT-4o, Claude, and Gemini exhibit highly correlated forecasting errors on 568 resolved binary prediction questions (mean pairwise error correlation r = 0.77, p < 0.001; r = 0.78 excluding likely-leaked questions), despite being developed independently by different organizations. In Study 2, we test whether this correlated bias has propagated into human crowd forecasts, using a within-question design that tracks community prediction shifts across the ChatGPT launch boundary (November 2022). We find that community forecasts move in the direction predicted by LLMs (r = 0.20, p = 0.007), but this shift is fully explained by rational updating toward ground truth. In Study 3, we examine whether the category-level pattern of human forecasting errors increasingly resembles the LLM bias fingerprint. We find the opposite: pre-ChatGPT human biases already strongly resembled the LLM pattern (r = 0.87), while post-ChatGPT the resemblance weakened (r = -0.28). Together, these findings reveal an epistemic monoculture that is built but not yet activated: three nominally independent AI systems share the same failure modes, amplifying precisely the biases humans already hold.