The bias of IID resampled backtests for rolling-window mean-variance portfolios

TL;DR

This paper examines the bias introduced by IID resampling in backtests of rolling-window mean-variance portfolios, revealing that the bias depends on return autocorrelation and can often be tolerable, but calls for structure-preserving methods.

Contribution

It identifies the bias mechanism in IID resampling for portfolio backtests, providing bounds and a heuristic to gauge bias severity, emphasizing the importance of structure-preserving resampling.

Findings

Bias depends on first-lag autocorrelation.

Resampling bias can be a fraction of estimation noise.

Disruption of temporal dependence can be tolerable.

Abstract

Backtests on historical data are the basis for practical evaluations of portfolio selection rules, but their reliability is often limited by reliance on a single sample path. This can lead to high estimation variance. Resampling techniques offer a potential solution by increasing the effective sample size, but can disrupt the temporal ordering inherent in financial data and introduce significant bias. This paper investigates the critical questions: First, How large is this bias for Sharpe Ratio estimates?, and then, second: What are its primary drivers?. We focus on the canonical rolling-window mean-variance portfolio rule. Our contributions are identifying the bias mechanism, and providing a practical heuristic for gauging bias severity. We show that the bias arises from the disruption of train-test dependence linked to the return auto-covariance structure and derive bounds for the bias which show a strong dependence on the observable first-lag autocorrelation. Using simulations to confirm these findings, it is revealed that the resulting Sharpe Ratio bias is often a fraction of a typical backtest's estimation noise, benefiting from partial offsetting of component biases. Empirical analysis further illustrates that differences between IID-resampled and standard backtests align qualitatively with these drivers. Surprisingly, our results suggest that while IID resampling can disrupt temporal dependence, its resulting bias can often be tolerable. However, we highlight the need for structure-preserving resampling methods.