Bootstrapping Generalization Error Bounds for Time Series

TL;DR

This paper develops bootstrap-based methods to construct valid confidence intervals for forecasting risk in stationary, ergodic time series, demonstrating their effectiveness through theoretical analysis and simulations.

Contribution

It introduces bootstrap confidence intervals for risk estimation in time series, applicable under mixing conditions, with validation for AR models and asymptotic independence results.

Findings

Bootstrap provides valid confidence intervals under mixing conditions.

Finite-sample coverage converges to the asymptotic level.

AR models satisfy regularity conditions even when mis-specified.

Abstract

We consider the problem of finding confidence intervals for the risk of forecasting the future of a stationary, ergodic stochastic process, using a model estimated from the past of the process. We show that a bootstrap procedure provides valid confidence intervals for the risk, when the data source is sufficiently mixing, and the loss function and the estimator are suitably smooth. Autoregressive (AR(d)) models estimated by least squares obey the necessary regularity conditions, even when mis-specified, and simulations show that the finite- sample coverage of our bounds quickly converges to the theoretical, asymptotic level. As an intermediate step, we derive sufficient conditions for asymptotic independence between empirical distribution functions formed by splitting a realization of a stochastic process, of independent interest.