Exact upper and lower bounds on the misclassification probability

TL;DR

This paper derives precise upper and lower bounds on the minimal misclassification probability for finite classes using total variation norms, and compares these bounds with existing entropy-based bounds.

Contribution

It introduces exact bounds based on total variation norms and compares them with prior entropy-based bounds, providing a new perspective on classification limits.

Findings

Bounds are expressed in terms of total variation norms.

Comparison shows differences with entropy-based bounds.

Provides exact bounds for finite class classification.

Abstract

Exact lower and upper bounds on the best possible misclassification probability for a finite number of classes are obtained in terms of the total variation norms of the differences between the sub-distributions over the classes. These bounds are compared with the exact bounds in terms of the conditional entropy obtained by Feder and Merhav.