Efficient learning of neighbor representations for boundary trees and forests

TL;DR

This paper presents Differentiable Boundary Sets, a scalable and efficient neighbor-based classification method that improves upon previous boundary tree algorithms by reducing training time and enhancing accuracy on image datasets.

Contribution

Introduction of Differentiable Boundary Sets, a novel algorithm that overcomes scalability issues of prior boundary tree methods and improves classification performance.

Findings

Reduces training time significantly.

Improves classification accuracy on MNIST and Fashion-MNIST.

Demonstrates scalability to large datasets.

Abstract

We introduce a semiparametric approach to neighbor-based classification. We build off the recently proposed Boundary Trees algorithm by Mathy et al.(2015) which enables fast neighbor-based classification, regression and retrieval in large datasets. While boundary trees use an Euclidean measure of similarity, the Differentiable Boundary Tree algorithm by Zoran et al.(2017) was introduced to learn low-dimensional representations of complex input data, on which semantic similarity can be calculated to train boundary trees. As is pointed out by its authors, the differentiable boundary tree approach contains a few limitations that prevents it from scaling to large datasets. In this paper, we introduce Differentiable Boundary Sets, an algorithm that overcomes the computational issues of the differentiable boundary tree scheme and also improves its classification accuracy and data representability. Our algorithm is efficiently implementable with existing tools and offers a significant reduction in training time. We test and compare the algorithms on the well known MNIST handwritten digits dataset and the newer Fashion-MNIST dataset by Xiao et al.(2017).