A Statistical Distance Derived From The Kolmogorov-Smirnov Test: specification, reference measures (benchmarks) and example uses

TL;DR

This paper introduces a normalized statistical distance based on the Kolmogorov-Smirnov test, providing benchmarks and demonstrating its robustness and utility in distinguishing between samples from different distributions.

Contribution

It defines a new normalized distance measure derived from the KS statistic, along with reference benchmarks and practical examples of its application.

Findings

The $c'$ statistic effectively measures differences between samples.

Benchmarks from standard distributions validate the measure's robustness.

Real data examples demonstrate practical utility and insights.

Abstract

Statistical distances quantifies the difference between two statistical constructs. In this article, we describe reference values for a distance between samples derived from the Kolmogorov-Smirnov statistic $D_{F,F'}$. Each measure of the $D_{F,F'}$ is a measure of difference between two samples. This distance is normalized by the number of observations in each sample to yield the $c'=D_{F,F'}\sqrt{\frac{n n'}{n+n'}}$ statistic, for which high levels favor the rejection of the null hypothesis (that the samples are drawn from the same distribution). One great feature of $c'$ is that it inherits the robustness of $D_{F,F'}$ and is thus suitable for use in settings where the underlying distributions are not known. Benchmarks are obtained by comparing samples derived from standard distributions. The supplied example applications of the $c'$ statistic for the distinction of samples in real data enables further insights about the robustness and power of such statistical distance.