Monte Carlo method for constructing confidence intervals with unconstrained and constrained nuisance parameters in the NOvA experiment

TL;DR

This paper discusses a Monte Carlo approach for constructing confidence intervals in neutrino oscillation experiments, addressing challenges with nuisance parameters where traditional methods like Wilks' theorem fail.

Contribution

It presents the profile construction method used by NOvA, including its implementation, validation, and practical considerations for handling nuisance parameters in complex experiments.

Findings

Validated the profile construction method through toy studies.

Provided practical guidelines for implementing Monte Carlo confidence intervals.

Addressed limitations of Wilks' theorem in neutrino experiments.

Abstract

Measuring observables to constrain models using maximum-likelihood estimation is fundamental to many physics experiments. Wilks' theorem provides a simple way to construct confidence intervals on model parameters, but it only applies under certain conditions. These conditions, such as nested hypotheses and unbounded parameters, are often violated in neutrino oscillation measurements and other experimental scenarios. Monte Carlo methods can address these issues, albeit at increased computational cost. In the presence of nuisance parameters, however, the best way to implement a Monte Carlo method is ambiguous. This paper documents the method selected by the NOvA experiment, the profile construction. It presents the toy studies that informed the choice of method, details of its implementation, and tests performed to validate it. It also includes some practical considerations which may be of use to others choosing to use the profile construction.