Exact model comparisons in the plausibility framework

TL;DR

This paper extends the plausibility framework for exact model testing by incorporating weighing, enabling efficient model comparisons with finite sample guarantees and applications to high-dimensional data.

Contribution

It introduces a weighted plausibility method that generalizes existing tests, providing asymptotic equivalence to LRT and finite sample control, with demonstrated applications in various data analyses.

Findings

Weighted plausibility is asymptotically equivalent to LRT.

Finite sample guarantees are maintained under the null hypothesis.

The method outperforms data-splitting procedures in simulations.

Abstract

Plausibility is a formalization of exact tests for parametric models and generalizes procedures such as Fisher's exact test. The resulting tests are based on cumulative probabilities of the probability density function and evaluate consistency with a parametric family while providing exact control of the $\alpha$ level for finite sample size. Model comparisons are inefficient in this approach. We generalize plausibility by incorporating weighing which allows to perform model comparisons. We show that one weighing scheme is asymptotically equivalent to the likelihood ratio test (LRT) and has finite sample guarantees for the test size under the null hypothesis unlike the LRT. We confirm theoretical properties in simulations that mimic the data set of our data application. We apply the method to a retinoblastoma data set and demonstrate a parent-of-origin effect. Weighted plausibility also has applications in high-dimensional data analysis and P-values for penalized regression models can be derived. We demonstrate superior performance as compared to a data-splitting procedure in a simulation study. We apply weighted plausibility to a high-dimensional gene expression, case-control prostate cancer data set. We discuss the flexibility of the approach by relating weighted plausibility to targeted learning, the bootstrap, and sparsity selection.