A Cautionary Tale of Decorrelating Theory Uncertainties

TL;DR

This paper warns that decorrelating theory uncertainties in machine learning classifiers can give a misleading impression of reduced uncertainty, emphasizing the need for caution without a full statistical decomposition.

Contribution

It provides explicit examples showing that decorrelation may underestimate true uncertainties in theory-based errors, highlighting potential pitfalls.

Findings

Decorrelating theory uncertainties can significantly underestimate actual uncertainties.

Examples include fragmentation modeling and higher-order correction uncertainties.

Caution is advised when applying decorrelation to non-statistical uncertainties.

Abstract

A variety of techniques have been proposed to train machine learning classifiers that are independent of a given feature. While this can be an essential technique for enabling background estimation, it may also be useful for reducing uncertainties. We carefully examine theory uncertainties, which typically do not have a statistical origin. We will provide explicit examples of two-point (fragmentation modeling) and continuous (higher-order corrections) uncertainties where decorrelating significantly reduces the apparent uncertainty while the actual uncertainty is much larger. These results suggest that caution should be taken when using decorrelation for these types of uncertainties as long as we do not have a complete decomposition into statistically meaningful components.