A Dimension-Independent discriminant between distributions

TL;DR

This paper introduces a dimension-independent method using a cross-match statistic to estimate Henze-Penrose divergence, providing a non-parametric way to bound Bayes error in classification tasks.

Contribution

It proposes a novel, dimension-independent estimator for Henze-Penrose divergence based on optimal weighted matching, improving upon previous methods.

Findings

The method accurately estimates divergence in simulations.

It effectively bounds Bayes error on real datasets.

The approach is computationally feasible for high-dimensional data.

Abstract

Henze-Penrose divergence is a non-parametric divergence measure that can be used to estimate a bound on the Bayes error in a binary classification problem. In this paper, we show that a cross-match statistic based on optimal weighted matching can be used to directly estimate Henze-Penrose divergence. Unlike an earlier approach based on the Friedman-Rafsky minimal spanning tree statistic, the proposed method is dimension-independent. The new approach is evaluated using simulation and applied to real datasets to obtain Bayes error estimates.