Distribution-Free Rates in Neyman-Pearson Classification

TL;DR

This paper characterizes the optimal distribution-free rates for Neyman-Pearson classification, revealing a dichotomy based on a geometric three-points-separation condition related to VC dimension.

Contribution

It provides a complete characterization of minimax rates in Neyman-Pearson classification across all distribution pairs, based on a novel geometric condition.

Findings

Identifies a dichotomy between hard and easy classes based on a three-points-separation condition.

Provides a full characterization of distribution-free minimax rates.

Relates the rates to VC dimension and geometric properties of the classifier class.

Abstract

We consider the problem of Neyman-Pearson classification which models unbalanced classification settings where error w.r.t. a distribution $\mu_1$ is to be minimized subject to low error w.r.t. a different distribution $\mu_0$. Given a fixed VC class $\mathcal{H}$ of classifiers to be minimized over, we provide a full characterization of possible distribution-free rates, i.e., minimax rates over the space of all pairs $(\mu_0, \mu_1)$. The rates involve a dichotomy between hard and easy classes $\mathcal{H}$ as characterized by a simple geometric condition, a three-points-separation condition, loosely related to VC dimension.