Statistical analysis of event classification in experimental data

TL;DR

This paper provides a comprehensive tutorial and rigorous statistical analysis of event classification in experimental data, focusing on Bayesian and frequentist methods for estimating signal and background components.

Contribution

It introduces a detailed tutorial on statistical event classification and offers a rigorous comparison of Bayesian and frequentist estimators for signal-background separation.

Findings

Bayesian and frequentist estimators are evaluated for accuracy.

Methods for estimating unknown signal and background events are presented.

Data quality metrics like purity and contamination are discussed.

Abstract

The paper addresses general aspects of experimental data analysis, dealing with the separation of ``signal vs. background''. It consists of two parts. Part I is a tutorial on statistical event classification, Bayesian inference, and test optimization. Aspects of the base data sample if being created by Poisson processes are discussed, and a method for estimating the unknown numbers of signal and background events is presented. Data quality of the selected events sample is assessed by the expected purity and background contamination. Part II contains a rigorous statistical analysis of the methods discussed in Part I. Both Bayesian and frequentist estimators of the unknown signal/background content are investigated. The estimates and their stochastic uncertainties are calculated for various conjugate priors in the Bayesian case, and for three choices of the virtual parent population in the frequentist case.