On the Limits of Prediction: Forecastability Profiles and Information Decay in Time Series

TL;DR

This paper uses information theory to precisely define forecastability in time series, revealing how predictive information varies with lead time and identifying horizons where meaningful prediction is possible.

Contribution

It introduces a formal framework for forecastability profiles and informative horizons, separating inherent data limits from modeling errors in time series prediction.

Findings

Forecastability is quantified by the mutual information between future and available data.

Forecastability profiles can be non-monotonic, highlighting seasonal or lag-specific predictability.

The framework identifies horizons where prediction gains are practically meaningful, guiding modeling efforts.

Abstract

Forecasting accuracy is bounded by the information available about the future. This paper makes that statement precise using information-theoretic tools. Under logarithmic loss, the expected performance of any probabilistic forecast decomposes into two parts: an irreducible component and an approximation component. The irreducible term is the conditional entropy of the future given the available information, while the approximation term is the divergence between the true conditional distribution and the forecasting method. The gap between this conditional-entropy limit and an unconditional baseline is exactly the mutual information between the future observation and the declared information set. This leads to a definition of forecastability as the maximum achievable reduction in expected log loss. Evaluated across horizons, forecastability forms a profile that describes how predictive information varies with lead time. This profile reflects the dependence structure of the process and need not be monotone: predictive information may be concentrated at particular lags, including seasonal horizons, even when intermediate horizons contain little useful signal. From this profile, the paper defines the informative horizon set: the horizons at which forecastability exceeds a practical threshold. At horizons not in this set, the achievable gain over the unconditional baseline is necessarily small, regardless of the forecasting method used. The framework therefore separates what is learnable from what is not, and distinguishes limits imposed by the data from errors introduced by modelling. The result is a pre-modelling diagnostic that identifies where meaningful prediction is feasible before any model is chosen, providing a principled basis for allocating modelling effort across forecast horizons.