Self-Calibrating the Look-Elsewhere Effect: Fast Evaluation of the Statistical Significance Using Peak Heights

TL;DR

This paper presents a fast, computationally inexpensive method to calibrate the statistical significance of peaks in likelihood analyses, effectively correcting for the look-elsewhere effect in large parameter space searches.

Contribution

It introduces a self-calibrating technique that uses peak heights to accurately estimate false alarm probabilities, improving efficiency over existing methods.

Findings

Method provides rapid significance calibration with negligible computational cost.

Applicable to diverse astronomy data analyses including planet detection and cosmology.

Enhances accuracy of significance estimates in large parameter searches.

Abstract

In experiments where one searches a large parameter space for an anomaly, one often finds many spurious noise-induced peaks in the likelihood. This is known as the look-elsewhere effect, and must be corrected for when performing statistical analysis. This paper introduces a method to calibrate the false alarm probability (FAP), or $p$-value, for a given dataset by considering the heights of the highest peaks in the likelihood. In the simplest form of self-calibration, the look-elsewhere-corrected $\chi^2$ of a physical peak is approximated by the $\chi^2$ of the peak minus the $\chi^2$ of the highest noise-induced peak. Generalizing this concept to consider lower peaks provides a fast method to quantify the statistical significance with improved accuracy. In contrast to alternative methods, this approach has negligible computational cost as peaks in the likelihood are a byproduct of every peak-search analysis. We apply to examples from astronomy, including planet detection, periodograms, and cosmology.