Intrinsic Dimension Adaptive Partitioning for Kernel Methods

TL;DR

This paper establishes optimal learning rates for kernel methods using data-dependent partitions based on the fractal dimension of the data support, and demonstrates their effectiveness through theoretical proofs and experiments.

Contribution

It introduces a new approach that adapts to the intrinsic fractal dimension of data, achieving optimal rates without prior knowledge of this dimension.

Findings

Optimal minimax learning rates are proven for the proposed methods.

Training validation can achieve these rates without prior dimension knowledge.

Methods generalize well from high-dimensional embeddings.

Abstract

We prove minimax optimal learning rates for kernel ridge regression, resp.~support vector machines based on a data dependent partition of the input space, where the dependence of the dimension of the input space is replaced by the fractal dimension of the support of the data generating distribution. We further show that these optimal rates can be achieved by a training validation procedure without any prior knowledge on this intrinsic dimension of the data. Finally, we conduct extensive experiments which demonstrate that our considered learning methods are actually able to generalize from a dataset that is non-trivially embedded in a much higher dimensional space just as well as from the original dataset.