Higher-Order Coverage Errors of Batching Methods via Edgeworth Expansions on $t$-Statistics

TL;DR

This paper investigates the higher-order coverage errors of batching methods using Edgeworth expansions on t-statistics, revealing non-monotonic effects of batch number and identifying the sectioned jackknife as superior with many batches.

Contribution

It develops algorithms to estimate higher-order coverage errors for batching methods and compares their performance both theoretically and numerically.

Findings

Coverage errors are non-monotonic in the number of batches.

Sectioned jackknife performs best with a large number of batches.

No batching method is uniformly best across all scenarios.

Abstract

While batching methods have been widely used in simulation and statistics, it is open regarding their higher-order coverage behaviors and whether one variant is better than the others in this regard. We develop techniques to obtain higher-order coverage errors for batching methods by building Edgeworth-type expansions on $t$-statistics. The coefficients in these expansions are intricate analytically, but we provide algorithms to estimate the coefficients of the $n^{-1}$ error term via Monte Carlo simulation. We provide insights on the effect of the number of batches on the coverage error where we demonstrate generally non-monotonic relations. We also compare different batching methods both theoretically and numerically, and argue that none of the methods is uniformly better than the others in terms of coverage. However, when the number of batches is large, sectioned jackknife has the best coverage among all.