On an Exact and Nonparametric Test for the Separability of Two Classes by Means of a Simple Threshold

TL;DR

This paper presents an exact, nonparametric statistical test to determine if a single threshold can separate two classes, useful for variable selection and robust classification in univariate settings.

Contribution

It introduces a novel test based on prediction error that is exact, nonparametric, and computationally efficient for finite samples, enhancing variable selection methods.

Findings

The test accurately identifies separability with finite sample guarantees.

It can incorporate operational conditions into the classification assessment.

The recursive algorithm efficiently computes the exact null distribution.

Abstract

This paper introduces a statistical test inferring whether a variable allows separating two classes by means of a single critical value. Its test statistic is the prediction error of a nonparametric threshold classifier. While this approach is adequate for univariate classification tasks, it is especially advantageous for filter-type variable selection. It constitutes a robust and nonparametric method which may identify important otherwise neglected variables. It can incorporate the operating conditions of the classification task. Last but not least, the exact finite sample distribution of the test statistic under the null hypothesis can be calculated using a fast recursive algorithm.