Generalizations related to hypothesis testing with the Posterior distribution of the Likelihood Ratio

TL;DR

This paper extends the Posterior distribution of the Likelihood Ratio (PLR) to composite hypotheses testing, establishing finite-sample reconciliation with frequentist p-values and proposing new Bayesian-based extensions.

Contribution

It introduces two novel extensions of the PLR for composite hypotheses, including cases with improper priors and a Bayesian Neyman-Pearson framework.

Findings

Finite-sample reconciliation between PLR and p-value for simple hypotheses

Extension of PLR to composite hypotheses with proper posterior

Introduction of a Bayesian Neyman-Pearson lemma-based discrepancy measure

Abstract

The Posterior distribution of the Likelihood Ratio (PLR) is proposed by Dempster in 1974 for significance testing in the simple vs composite hypotheses case. In this hypotheses test case, classical frequentist and Bayesian hypotheses tests are irreconcilable, as emphasized by Lindley's paradox, Berger & Selke in 1987 and many others. However, Dempster shows that the PLR (with inner threshold 1) is equal to the frequentist p-value in the simple Gaussian case. In 1997, Aitkin extends this result by adding a nuisance parameter and showing its asymptotic validity under more general distributions. Here we extend the reconciliation between the PLR and a frequentist p-value for a finite sample, through a framework analogous to the Stein's theorem frame in which a credible (Bayesian) domain is equal to a confidence (frequentist) domain. This general reconciliation result only concerns simple vs composite hypotheses testing. The measures proposed by Aitkin in 2010 and Evans in 1997 have interesting properties and extend Dempster's PLR but only by adding a nuisance parameter. Here we propose two extensions of the PLR concept to the general composite vs composite hypotheses test. The first extension can be defined for improper priors as soon as the posterior is proper. The second extension appears from a new Bayesian-type Neyman-Pearson lemma and emphasizes, from a Bayesian perspective, the role of the LR as a discrepancy variable for hypothesis testing.