Online Consistency of the Nearest Neighbor Rule

TL;DR

This paper proves that the nearest neighbor rule is online consistent in doubling metric spaces under mild assumptions, extending its known consistency beyond traditional statistical or geometric conditions.

Contribution

It establishes online consistency of the nearest neighbor rule for all measurable functions in doubling metric spaces under mild distributional assumptions.

Findings

Nearest neighbor rule is online consistent in doubling metric spaces.

Consistency holds under mild assumptions of uniform absolute continuity.

Extends known consistency results beyond i.i.d. and well-separated classes.

Abstract

In the realizable online setting, a learner is tasked with making predictions for a stream of instances, where the correct answer is revealed after each prediction. A learning rule is online consistent if its mistake rate eventually vanishes. The nearest neighbor rule (Fix and Hodges, 1951) is a fundamental prediction strategy, but it is only known to be consistent under strong statistical or geometric assumptions: the instances come i.i.d. or the label classes are well-separated. We prove online consistency for all measurable functions in doubling metric spaces under the mild assumption that the instances are generated by a process that is uniformly absolutely continuous with respect to a finite, upper doubling measure.