A method for comparing non-nested models with application to astrophysical searches for new physics

TL;DR

This paper introduces a new statistical method for comparing non-nested models, crucial for astrophysical data analysis such as dark matter searches, where traditional tests are not applicable.

Contribution

The authors develop and validate a novel frequentist approach for non-nested model comparison, applicable to astrophysical data analysis.

Findings

Method is simple and generally applicable.

Validated through simulation studies.

Effective in astrophysical model comparisons.

Abstract

Searches for unknown physics and decisions between competing astrophysical models to explain data both rely on statistical hypothesis testing. The usual approach in searches for new physical phenomena is based on the statistical Likelihood Ratio Test (LRT) and its asymptotic properties. In the common situation, when neither of the two models under comparison is a special case of the other i.e., when the hypotheses are non-nested, this test is not applicable. In astrophysics, this problem occurs when two models that reside in different parameter spaces are to be compared. An important example is the recently reported excess emission in astrophysical $\gamma$-rays and the question whether its origin is known astrophysics or dark matter. We develop and study a new, simple, generally applicable, frequentist method and validate its statistical properties using a suite of simulations studies. We exemplify it on realistic simulated data of the Fermi-LAT $\gamma$-ray satellite, where non-nested hypotheses testing appears in the search for particle dark matter.