The Likelihood Ratio Test and Full Bayesian Significance Test under small sample sizes for contingency tables

TL;DR

This paper evaluates the performance of likelihood ratio and Bayesian significance tests in small sample contingency tables, finding that asymptotic-based tests perform well even with limited data.

Contribution

It introduces an accurate small-sample P-value for these tests and compares its behavior to traditional asymptotic methods across various scenarios.

Findings

Asymptotic tests perform well even with small samples.

Exact Fisher test reduces sample space for 2x2 tables.

All tested indices show similar behavior in small samples.

Abstract

Hypothesis testing in contingency tables is usually based on asymptotic results, thereby restricting its proper use to large samples. To study these tests in small samples, we consider the likelihood ratio test and define an accurate index, the P-value, for the celebrated hypotheses of homogeneity, independence, and Hardy-Weinberg equilibrium. The aim is to understand the use of the asymptotic results of the frequentist Likelihood Ratio Test and the Bayesian FBST -- Full Bayesian Significance Test -- under small-sample scenarios. The proposed exact P-value is used as a benchmark to understand the other indices. We perform analysis in different scenarios, considering different sample sizes and different table dimensions. The exact Fisher test for $2 \times 2$ tables that drastically reduces the sample space is also discussed. The main message of this paper is that all indices have very similar behavior, so the tests based on asymptotic results are very good to be used in any circumstance, even with small sample sizes.