ABCDisCo: Automating the ABCD Method with Machine Learning

TL;DR

This paper introduces ABCDisCo, a machine learning approach to automate and optimize the ABCD background estimation method in high energy physics, improving accuracy and background rejection.

Contribution

It presents a novel machine learning framework that automates classifier design for the ABCD method, enhancing its performance and reliability.

Findings

Automated classifiers improve ABCD closure.

Enhanced background rejection achieved.

Reduced signal contamination in control regions.

Abstract

The ABCD method is one of the most widely used data-driven background estimation techniques in high energy physics. Cuts on two statistically-independent classifiers separate signal and background into four regions, so that background in the signal region can be estimated simply using the other three control regions. Typically, the independent classifiers are chosen "by hand" to be intuitive and physically motivated variables. Here, we explore the possibility of automating the design of one or both of these classifiers using machine learning. We show how to use state-of-the-art decorrelation methods to construct powerful yet independent discriminators. Along the way, we uncover a previously unappreciated aspect of the ABCD method: its accuracy hinges on having low signal contamination in control regions not just overall, but relative to the signal fraction in the signal region. We demonstrate the method with three examples: a simple model consisting of three-dimensional Gaussians; boosted hadronic top jet tagging; and a recasted search for paired dijet resonances. In all cases, automating the ABCD method with machine learning significantly improves performance in terms of ABCD closure, background rejection and signal contamination.