Breaking the curse of dimensionality with Isolation Kernel

TL;DR

This paper demonstrates that the Isolation Kernel can effectively break the curse of dimensionality across various machine learning tasks, outperforming traditional kernels in high-dimensional spaces.

Contribution

It introduces the Isolation Kernel as a novel approach capable of overcoming the curse of dimensionality, supported by theoretical analysis and extensive empirical validation.

Findings

Isolation Kernel outperforms other kernels in high-dimensional tasks

It maintains consistent performance in clustering, classification, and visualization

Theoretical proof shows Isolation Kernel uniquely breaks the curse

Abstract

The curse of dimensionality has been studied in different aspects. However, breaking the curse has been elusive. We show for the first time that it is possible to break the curse using the recently introduced Isolation Kernel. We show that only Isolation Kernel performs consistently well in indexed search, spectral & density peaks clustering, SVM classification and t-SNE visualization in both low and high dimensions, compared with distance, Gaussian and linear kernels. This is also supported by our theoretical analyses that Isolation Kernel is the only kernel that has the provable ability to break the curse, compared with existing metric-based Lipschitz continuous kernels.