Speculate-Correct Error Bounds for k-Nearest Neighbor Classifiers

TL;DR

This paper introduces a speculate-correct method to derive exponential error bounds for k-nearest neighbor classifiers, showing they have predictable error rates despite complex decision boundaries.

Contribution

The paper presents a novel speculate-correct approach to establish error bounds for local classifiers like k-NN, with explicit bounds for finite samples.

Findings

k-NN classifiers have exponential error bounds

Error rate is O(sqrt((k + ln n) / n)) for n samples

Decision boundaries are complex but bounds are predictable

Abstract

We introduce the speculate-correct method to derive error bounds for local classifiers. Using it, we show that k nearest neighbor classifiers, in spite of their famously fractured decision boundaries, have exponential error bounds with O(sqrt((k + ln n) / n)) error bound range for n in-sample examples.