Statistical coverage for supersymmetric parameter estimation: a case study with direct detection of dark matter

TL;DR

This paper assesses the statistical coverage of confidence intervals in supersymmetric dark matter models, revealing issues with under- and over-coverage depending on priors and sampling methods, and compares frequentist and Bayesian approaches.

Contribution

It provides a detailed analysis of the coverage properties of confidence intervals in the CMSSM using nested sampling, highlighting the influence of priors and sampling effects on statistical validity.

Findings

Both under- and over-coverage observed depending on priors and benchmarks.

Sampling effects and priors significantly influence coverage properties.

Bayesian credible intervals show notable under-coverage.

Abstract

Models of weak-scale supersymmetry offer viable dark matter (DM) candidates. Their parameter spaces are however rather large and complex, such that pinning down the actual parameter values from experimental data can depend strongly on the employed statistical framework and scanning algorithm. In frequentist parameter estimation, a central requirement for properly constructed confidence intervals is that they cover true parameter values, preferably at exactly the stated confidence level when experiments are repeated infinitely many times. Since most widely-used scanning techniques are optimised for Bayesian statistics, one needs to assess their abilities in providing correct confidence intervals in terms of the statistical coverage. Here we investigate this for the Constrained Minimal Supersymmetric Standard Model (CMSSM) when only constrained by data from direct searches for dark matter. We construct confidence intervals from one-dimensional profile likelihoods and study the coverage by generating several pseudo-experiments for a few benchmark sets of pseudo-true parameters. We use nested sampling to scan the parameter space and evaluate the coverage for the benchmarks when either flat or logarithmic priors are imposed on gaugino and scalar mass parameters. The sampling algorithm has been used in the configuration usually adopted for exploration of the Bayesian posterior. We observe both under- and over-coverage, which in some cases vary quite dramatically when benchmarks or priors are modified. We show how most of the variation can be explained as the impact of explicit priors as well as sampling effects, where the latter are indirectly imposed by physicality conditions. For comparison, we also evaluate the coverage for Bayesian credible intervals, and observe significant under-coverage in those cases.