Partially Bayes p-values for large scale inference

TL;DR

This paper introduces partially Bayes p-values for large-scale inference, combining Bayesian modeling of nuisance parameters with fixed primary parameters, improving calibration and power in complex models.

Contribution

It develops a novel partially Bayesian p-value methodology that models nuisance parameters hierarchically and avoids data double use, enhancing inference in large-scale problems.

Findings

Improved calibration over frequentist methods.

Enhanced power in normal means and location-scale models.

Effective use of Dirichlet processes and Pólya trees.

Abstract

We seek to conduct statistical inference for a large collection of primary parameters, each with its own nuisance parameters. Our approach is partially Bayesian, in that we treat the primary parameters as fixed while we model the nuisance parameters as random and drawn from an unknown distribution which we endow with a nonparametric prior. We compute partially Bayes p-values by conditioning on nuisance parameter statistics, that is, statistics that are ancillary for the primary parameters and informative about the nuisance parameters. The proposed p-values have a Bayesian interpretation as tail areas computed with respect to the posterior distribution of the nuisance parameters. Similarly to the conditional predictive p-values of Bayarri and Berger, the partially Bayes p-values avoid double use of the data (unlike posterior predictive p-values). A key ingredient of our approach is that we model nuisance parameters hierarchically across problems; the sharing of information across problems leads to improved calibration. We illustrate the proposed partially Bayes p-values in two applications: the normal means problem with unknown variances and a location-scale model with unknown distribution shape. We model the scales via Dirichlet processes in both examples and the distribution shape via P\'olya trees in the second. Our proposed partially Bayes p-values increase power and calibration compared to purely frequentist alternatives.