Refinement revisited with connections to Bayes error, conditional entropy and calibrated classifiers

TL;DR

This paper revisits the concept of refinement in probability elicitation, connecting it to Bayes error, entropy, and classifier calibration, providing new bounds and insights into proper scoring rules.

Contribution

It introduces a Hilbert space interpretation of refinement, links it to maximal marginal diversity and conditional entropy, and derives tight bounds on Bayes error, also relating to classifier calibration.

Findings

Refinement bounds on Bayes error using entropy measures.

Connections established between refinement, calibrated classifiers, and margin losses.

Reformulation of refinement in data distribution and classifier output settings.

Abstract

The concept of refinement from probability elicitation is considered for proper scoring rules. Taking directions from the axioms of probability, refinement is further clarified using a Hilbert space interpretation and reformulated into the underlying data distribution setting where connections to maximal marginal diversity and conditional entropy are considered and used to derive measures that provide arbitrarily tight bounds on the Bayes error. Refinement is also reformulated into the classifier output setting and its connections to calibrated classifiers and proper margin losses are established.