Goodness of fit tests in terms of local levels with special emphasis on higher criticism tests

TL;DR

This paper introduces the concept of local levels to analyze goodness of fit tests, focusing on the asymptotic behavior of local levels in Kolmogorov-Smirnov and higher criticism tests, and explores constructing new tests based on this framework.

Contribution

It provides a novel perspective on GOF tests through local levels, characterizes their asymptotic behavior, and offers insights for developing new tests with desired sensitivity properties.

Findings

Local levels of KS tests remain above zero asymptotically.

Local levels of HC tests tend to zero as sample size increases.

No asymptotic level  GOF test has all local levels bounded away from zero.

Abstract

Instead of defining goodness of fit (GOF) tests in terms of their test statistics, we present an alternative method by introducing the concept of local levels, which indicate high or low local sensitivity of a test. Local levels can act as a starting point for the construction of new GOF tests. We study the behavior of local levels when applied to some well-known GOF tests such as Kolmogorov-Smirnov (KS) tests, higher criticism (HC) tests and tests based on phi-divergences. The main focus is on a rigorous characterization of the asymptotic behavior of local levels of the original HC tests which leads to several further asymptotic results for local levels of other GOF tests including GOF tests with equal local levels. While local levels of KS tests, which are related to the central range, are asymptotically strictly larger than zero, all local levels of HC tests converge to zero as the sample size increases. Consequently, there exists no asymptotic level $\alpha$ GOF test such that all local levels are asymptotically bounded away from zero. Finally, by means of numerical computations we compare classical KS and HC tests to a GOF test with equal local levels.