A New ECDF Two-Sample Test Statistic

TL;DR

This paper introduces a new ECDF-based two-sample test statistic that offers higher power than existing tests, along with a practical R implementation and simulation results demonstrating its effectiveness.

Contribution

A novel ECDF two-sample test statistic with improved power and a computationally efficient R package implementation.

Findings

The new test outperforms existing ECDF-based tests in power.

The R package enables efficient finite-sample p-value computation.

Simulation studies confirm the test's superior performance.

Abstract

Empirical cumulative distribution functions (ECDFs) have been used to test the hypothesis that two samples come from the same distribution since the seminal contribution by Kolmogorov and Smirnov. This paper describes a statistic which is usable under the same conditions as Kolmogorov-Smirnov, but provides more power than other extant tests in that vein. I demonstrate a valid (conservative) procedure for producing finite-sample p-values. I outline the close relationship between this statistic and its two main predecessors. I also provide a public R package (CRAN: twosamples [2018]) implementing the testing procedure in $O(N\log(N))$ time with $O(N)$ memory. Using the package's functions, I perform several simulation studies showing the power improvements.