Noise Titration: Exact Distributional Benchmarking for Probabilistic Time Series Forecasting

TL;DR

This paper introduces a novel benchmarking approach for probabilistic time series forecasting that uses exact distributional inference through noise titration, enabling rigorous evaluation of models under controlled non-stationarity and noise conditions.

Contribution

It proposes a new interventionist benchmarking framework and extends the Fern architecture to natively output calibrated joint covariance structures, improving evaluation precision.

Findings

State-of-the-art models fail under non-stationary shifts and high noise.

Fern captures invariant measures and multivariate geometry of dynamics.

The framework enables exact negative log-likelihood evaluation.

Abstract

Modern time series forecasting is evaluated almost entirely through passive observation of single historical trajectories, rendering claims about a model's robustness to non-stationarity fundamentally unfalsifiable. We propose a paradigm shift toward interventionist, exact-statistical benchmarking. By systematically titrating calibrated Gaussian observation noise into known chaotic and stochastic dynamical systems, we transform forecasting from a black-box sequence matching game into an exact distributional inference task. Because the underlying data-generating process and noise variance are mathematically explicit, evaluation can rely on exact negative log-likelihoods and calibrated distributional tests rather than heuristic approximations. To fully leverage this framework, we extend the Fern architecture into a probabilistic generative model that natively parameterizes the Symmetric Positive Definite (SPD) cone, outputting calibrated joint covariance structures without the computational bottleneck of generic Jacobian modeling. Under this rigorous evaluation, we find that state-of-the-art zero-shot foundation models behave consistently with the context-parroting mechanism, failing systematically under non-stationary regime shifts and elevated noise. In contrast, Fern explicitly captures the invariant measure and multivariate geometry of the underlying dynamics, maintaining structural fidelity and statistically sharp calibration precisely where massive sequence-matching models collapse.