Learning by Unsupervised Nonlinear Diffusion

TL;DR

This paper introduces LUND, a clustering algorithm that uses diffusion geometry and density estimation to identify clusters with nonlinear shapes, outperforming spectral and density-based methods.

Contribution

The paper presents a novel diffusion-based clustering method that leverages diffusion time as a scale parameter, with theoretical analysis and empirical validation.

Findings

LUND accurately identifies clusters with nonlinear shapes.

Diffusion time reveals mesoscopic equilibria between clusters.

LUND outperforms spectral and density-based clustering methods.

Abstract

This paper proposes and analyzes a novel clustering algorithm that combines graph-based diffusion geometry with techniques based on density and mode estimation. The proposed method is suitable for data generated from mixtures of distributions with densities that are both multimodal and have nonlinear shapes. A crucial aspect of this algorithm is the use of time of a data-adapted diffusion process as a scale parameter that is different from the local spatial scale parameter used in many clustering algorithms. We prove estimates for the behavior of diffusion distances with respect to this time parameter under a flexible nonparametric data model, identifying a range of times in which the mesoscopic equilibria of the underlying process are revealed, corresponding to a gap between within-cluster and between-cluster diffusion distances. These structures can be missed by the top eigenvectors of the graph Laplacian, commonly used in spectral clustering. This analysis is leveraged to prove sufficient conditions guaranteeing the accuracy of the proposed \emph{learning by unsupervised nonlinear diffusion (LUND)} procedure. We implement LUND and confirm its theoretical properties on illustrative datasets, demonstrating the theoretical and empirical advantages over both spectral clustering and density-based clustering techniques.