On the Unknowable Limits to Prediction

TL;DR

This paper introduces a framework for understanding the limits of prediction accuracy by decomposing errors into epistemic and aleatoric components, emphasizing that some unpredictable outcomes can become predictable with better data and models.

Contribution

It provides a rigorous decomposition of predictive error and demonstrates how improving data quality and modeling can reduce 'irreducible' error in prediction tasks.

Findings

Distinguishes between aleatoric and epistemic error components.

Shows that predictive accuracy can improve with better data and algorithms.

Provides a framework for advancing computational predictive research.

Abstract

We propose a rigorous decomposition of predictive error, highlighting that not all 'irreducible' error is genuinely immutable. Many domains stand to benefit from iterative enhancements in measurement, construct validity, and modeling. Our approach demonstrates how apparently 'unpredictable' outcomes can become more tractable with improved data (across both target and features) and refined algorithms. By distinguishing aleatoric from epistemic error, we delineate how accuracy may asymptotically improve--though inherent stochasticity may remain--and offer a robust framework for advancing computational research.