Empirical complexity of comparator-based nearest neighbor descent

TL;DR

This paper presents an empirical analysis of a parallel Java implementation of the K-nearest neighbor descent algorithm, demonstrating its complexity, convergence behavior, and accuracy across different data distributions and dimensions.

Contribution

It introduces a parallel Java implementation with a natural termination criterion and provides empirical complexity and accuracy results for various data types.

Findings

Complexity is approximately O(n K^2 log_K(n)) for studied examples.

Number of update rounds is bounded by twice the diameter of the underlying graph.

Accuracy remains high up to 20 dimensions, then declines as predicted by theory.

Abstract

A Java parallel streams implementation of the $K$-nearest neighbor descent algorithm is presented using a natural statistical termination criterion. Input data consist of a set $S$ of $n$ objects of type V, and a Function<V, Comparator<V>>, which enables any $x \in S$ to decide which of $y, z \in S\setminus\{x\}$ is more similar to $x$. Experiments with the Kullback-Leibler divergence Comparator support the prediction that the number of rounds of $K$-nearest neighbor updates need not exceed twice the diameter of the undirected version of a random regular out-degree $K$ digraph on $n$ vertices. Overall complexity was $O(n K^2 \log_K(n))$ in the class of examples studied. When objects are sampled uniformly from a $d$-dimensional simplex, accuracy of the $K$-nearest neighbor approximation is high up to $d = 20$, but declines in higher dimensions, as theory would predict.