PHYSTAT Informal Review: Marginalizing versus Profiling of Nuisance Parameters

TL;DR

This paper compares Bayesian and frequentist methods for handling nuisance parameters in statistical models, highlighting their differences, applications, and limitations, especially in high-dimensional or complex scenarios.

Contribution

It provides an informal review of marginalization versus profiling approaches, including historical context, practical examples, and recent advanced methods in the statistics literature.

Findings

In regular models, approaches are asymptotically equivalent.

Different methods can yield different tests and confidence intervals outside regular models.

Extreme cases with many nuisance parameters challenge both approaches.

Abstract

This is a writeup, with some elaboration, of the talks by the two authors (a physicist and a statistician) at the first PHYSTAT Informal review on January 24, 2024. We discuss Bayesian and frequentist approaches to dealing with nuisance parameters, in particular, integrated versus profiled likelihood methods. In regular models, with finitely many parameters and large sample sizes, the two approaches are asymptotically equivalent. But, outside this setting, the two methods can lead to different tests and confidence intervals. Assessing which approach is better generally requires comparing the power of the tests or the length of the confidence intervals. This analysis has to be conducted on a case-by-case basis. In the extreme case where the number of nuisance parameters is very large, possibly infinite, neither approach may be useful. Part I provides an informal history of usage in high energy particle physics, including a simple illustrative example. Part II includes an overview of some more recently developed methods in the statistics literature, including methods applicable when the use of the likelihood function is problematic.