Error estimation and reduction with cross correlations

TL;DR

This paper discusses how cross correlations in Monte Carlo data affect error estimation and proposes methods using jackknife resampling and covariance analysis to improve accuracy and reduce variance in estimates.

Contribution

It introduces a straightforward approach to account for cross correlations and formulates optimal estimators with reduced variance using covariance analysis.

Findings

Properly accounting for cross correlations improves error estimates.

Jackknife resampling effectively avoids systematic errors.

Optimal estimators significantly reduce variance.

Abstract

Besides the well-known effect of autocorrelations in time series of Monte Carlo simulation data resulting from the underlying Markov process, using the same data pool for computing various estimates entails additional cross correlations. This effect, if not properly taken into account, leads to systematically wrong error estimates for combined quantities. Using a straightforward recipe of data analysis employing the jackknife or similar resampling techniques, such problems can be avoided. In addition, a covariance analysis allows for the formulation of optimal estimators with often significantly reduced variance as compared to more conventional averages.