Robust test statistics for data sets with missing correlation information

TL;DR

This paper introduces robust test statistics designed for datasets lacking correlation information, ensuring accurate or conservative uncertainty estimates whether correlations are absent, unknown, or perfect.

Contribution

It proposes new test statistics that are exact without correlation, conservative with unknown correlations, and exact with perfect correlations, addressing a common challenge in experimental data analysis.

Findings

Test statistics are exact with no correlation

They are conservative when correlations are unknown

They are exact in the case of perfect correlation

Abstract

Not all experiments publish their results with a description of the correlations between the data points. This makes it difficult to do hypothesis tests or model fits with that data, since just assuming no correlation can lead to an over- or underestimation of the resulting uncertainties. This work presents robust test statistics that can be used with data sets with missing correlation information. They are exact in the case of no correlation and either guaranteed to be conservative -- i.e. the uncertainty is never underestimated -- in the presence of correlations, or they are also exact in the degenerate case of perfect correlation between the data points.