Minimax rates for cost-sensitive learning on manifolds with approximate nearest neighbours

TL;DR

This paper establishes the optimal learning rates for cost-sensitive classification on manifolds using approximate nearest neighbors, extending classic results and demonstrating the effectiveness of low-dimensional projections.

Contribution

It generalizes minimax rates for cost-sensitive learning on manifolds and proves these rates are achieved by approximate nearest neighbor algorithms with random projections.

Findings

Minimax rates are derived for cost-sensitive classification on manifolds.

Approximate nearest neighbor algorithms attain these minimax rates with low-dimensional projections.

The required projection dimension depends on manifold reach, dimension, and marginal regularity.

Abstract

We study the approximate nearest neighbour method for cost-sensitive classification on low-dimensional manifolds embedded within a high-dimensional feature space. We determine the minimax learning rates for distributions on a smooth manifold, in a cost-sensitive setting. This generalises a classic result of Audibert and Tsybakov. Building upon recent work of Chaudhuri and Dasgupta we prove that these minimax rates are attained by the approximate nearest neighbour algorithm, where neighbours are computed in a randomly projected low-dimensional space. In addition, we give a bound on the number of dimensions required for the projection which depends solely upon the reach and dimension of the manifold, combined with the regularity of the marginal.

Figures

Full text

See pages 1-last of ReeveCostSensitiveALT17.pdf