Tables with Critical Values for the Meta-Analysis of Genuine and Fake $\boldsymbol{p}$-Values

TL;DR

This paper provides tables with critical values for meta-analysis tests that account for the presence of fake $p$-values, addressing issues of publication bias and unknown fake $p$-value proportions.

Contribution

It introduces practical tables of critical values for combined tests that consider fake $p$-values, improving meta-analysis robustness.

Findings

Tables enable more accurate meta-analysis with fake $p$-values.

Addresses the lack of closed-form distributions for combined test statistics.

Provides tools for handling publication bias in $p$-value meta-analysis.

Abstract

The classical theory for the meta-analysis of $p$-values is based on the assumption that if the overall null hypothesis is true, then all $p$-values used in a chosen combined test statistic are genuine, i.e., are observations from independent and identically distributed standard uniform random variables. However, the pressure felt by most researchers to publish, which is worsen by publication bias, can originate fake $p$-values to be reported, usually Beta(1,2) distributed. In general, the existence of fake $p$-values in a sample of $p$-values to be combined is unknown, and if, for some reason, there is information that they do exist, their number will most likely be unknown as well. Moreover, even if fake $p$-values are accounted for, the cumulative distribution function of classical combined test statistics does not have a closed-form expression that facilitates its practical usage. To overcome this problem, tables with estimated critical values are supplied for the commonly used combined tests for the meta-analysis of $p$-values when a few of them are fake ones, i.e., Beta(1,2) distributed.