AR-Sieve Bootstrap for the Random Forest and a simulation-based comparison with rangerts time series prediction

TL;DR

This paper introduces an AR-Sieve Bootstrap method for Random Forests in time series prediction, demonstrating improved accuracy over traditional bootstrap methods through simulation studies, with some efficiency trade-offs.

Contribution

It proposes combining RF with AR-Sieve Bootstrap for better time series prediction, addressing limitations of existing bootstrap strategies.

Findings

AR-Sieve Bootstrap increases variation among trees.

RF with ARSB outperforms other bootstrap methods in accuracy.

Improvements come with some efficiency costs.

Abstract

The Random Forest (RF) algorithm can be applied to a broad spectrum of problems, including time series prediction. However, neither the classical IID (Independent and Identically distributed) bootstrap nor block bootstrapping strategies (as implemented in rangerts) completely account for the nature of the Data Generating Process (DGP) while resampling the observations. We propose the combination of RF with a residual bootstrapping technique where we replace the IID bootstrap with the AR-Sieve Bootstrap (ARSB), which assumes the DGP to be an autoregressive process. To assess the new model's predictive performance, we conduct a simulation study using synthetic data generated from different types of DGPs. It turns out that ARSB provides more variation amongst the trees in the forest. Moreover, RF with ARSB shows greater accuracy compared to RF with other bootstrap strategies. However, these improvements are achieved at some efficiency costs.