Adaptive Metric Dimensionality Reduction

TL;DR

This paper introduces an adaptive, data-dependent approach to dimensionality reduction in metric spaces, providing theoretical bounds and an efficient algorithm analogous to PCA, to improve supervised learning tasks.

Contribution

It offers a new generalization bound for Lipschitz functions in nearly doubling metric spaces and proposes an efficient PCA-like algorithm for intrinsic dimension approximation.

Findings

Generalization bounds for Lipschitz functions in doubling metric spaces

An efficient algorithm approximating data's intrinsic dimension

Enhanced efficiency and generalization in supervised learning

Abstract

We study adaptive data-dependent dimensionality reduction in the context of supervised learning in general metric spaces. Our main statistical contribution is a generalization bound for Lipschitz functions in metric spaces that are doubling, or nearly doubling. On the algorithmic front, we describe an analogue of PCA for metric spaces: namely an efficient procedure that approximates the data's intrinsic dimension, which is often much lower than the ambient dimension. Our approach thus leverages the dual benefits of low dimensionality: (1) more efficient algorithms, e.g., for proximity search, and (2) more optimistic generalization bounds.