Signal discovery, limits, and uncertainties with sparse On/Off measurements: an objective Bayesian analysis

TL;DR

This paper introduces an objective Bayesian method for analyzing sparse On/Off measurements, providing a unified way to assess signal significance, strength, and uncertainties applicable even with zero counts, useful in physics and astrophysics.

Contribution

The paper presents a novel Bayesian approach that analytically derives the probability of background-only hypothesis and the posterior for the signal, applicable to sparse data with zero counts.

Findings

Analytical derivation of background hypothesis probability.

Unified calculation of signal significance and upper limits.

Method validated on gamma-ray burst data.

Abstract

For decades researchers have studied the On/Off counting problem, where a measured rate consists of two parts. One due to a signal process and another due to a background process, of which both magnitudes are unknown. While most frequentist methods are adequate for large count numbers, they cannot be applied to sparse data. Here I want to present a new objective Bayesian solution that only depends on three parameters: the number of events in the signal region, the number of events in the background region, and the ratio of the exposure for both regions. First, the probability of the hypothesis that the counts are due to background only is derived analytically. Second, the marginalized posterior for the signal parameter is also derived analytically. With this two-step approach it is easy to calculate the signal's significance, strength, uncertainty, or upper limit in a unified way. The approach is valid without restrictions for any count number including zero and may be widely applied in particle physics, cosmic-ray physics and high-energy astrophysics. In order to demonstrate its performance I apply the method to gamma-ray burst data.