Simulation Studies For Goodness-of-Fit and Two-Sample Methods For Univariate Data

TL;DR

This paper evaluates the power of various goodness-of-fit and two-sample tests for univariate data through extensive simulations, identifying a small set of methods that perform well across different scenarios.

Contribution

It provides comprehensive simulation results and recommends a small set of robust methods for goodness-of-fit and two-sample testing in univariate data.

Findings

No single test is universally best across all scenarios.

A small set of methods can reliably perform well.

Simulations cover both continuous and discrete data.

Abstract

We present the results of a large number of simulation studies regarding the power of various goodness-of-fit as well as nonparametric two-sample tests for univariate data. This includes both continuous and discrete data. In general no single method can be relied upon to provide good power, any one method may be quite good for some combination of null hypothesis and alternative and may fail badly for another. Based on the results of these studies we propose a fairly small number of methods chosen such that for any of the case studies included here at least one of the methods has good power. The studies were carried out using the R packages R2sample and Rgof, available from CRAN.