Effective sample size estimation based on concordance between p-value and posterior probability of the null hypothesis

TL;DR

This paper introduces a novel method for estimating the effective sample size (ESS) using p-values within a hypothesis testing framework, addressing prior-likelihood discordance and eliminating the need for baseline priors.

Contribution

It proposes a new ESS estimation approach based on p-values that accounts for prior-likelihood disconcordance and supports multiple priors without requiring a baseline prior.

Findings

Demonstrates superior performance of the p-value ESS method in simulations

Shows effectiveness of informative priors in eQTL data analysis

Enables negative ESS values when prior and likelihood are in disaccordance

Abstract

Estimating the effective sample size (ESS) of a prior distribution is an age-old yet pivotal challenge, with great implications for clinical trials and various biomedical applications. Although numerous endeavors have been dedicated to this pursuit, most of them neglect the likelihood context in which the prior is embedded, thereby considering all priors as "beneficial". In the limited studies of addressing harmful priors, specifying a baseline prior remains an indispensable step. In this paper, by means of the elegant bridge between the p-value and the posterior probability of the null hypothesis, we propose a new ESS estimation method based on p-value in the framework of hypothesis testing, expanding the scope of existing ESS estimation methods in three key aspects: (i) We address the specific likelihood context of the prior, enabling the possibility of negative ESS values in case of prior-likelihood disconcordance; (ii) By leveraging the well-established bridge between the frequentist and Bayesian configurations under noninformative priors, there is no need to specify a baseline prior which incurs another criticism of subjectivity; (iii) By incorporating ESS into the hypothesis testing framework, our $p$-value ESS estimation method transcends the conventional one-ESS-one-prior paradigm and accommodates one-ESS-multiple-priors paradigm, where the sole ESS may reflect the collaborative impact of multiple priors in diverse contexts. Through comprehensive simulation analyses, we demonstrate the superior performance of the p-value ESS estimation method in comparison with existing approaches. Furthermore, by applying this approach to an expression quantitative trait loci (eQTL) data analysis, we show the effectiveness of informative priors in uncovering gene eQTL loci.