Diffusion-based Semi-supervised Spectral Algorithm for Regression on Manifolds

TL;DR

This paper presents a diffusion-based spectral algorithm for semi-supervised regression on high-dimensional data embedded in lower-dimensional manifolds, leveraging graph Laplacian approximation and intrinsic manifold properties.

Contribution

It introduces a novel, data-driven semi-supervised spectral regression algorithm that adapts to manifold structures without predefined kernel functions.

Findings

Achieves convergence rate depending only on the manifold's intrinsic dimension

Effectively utilizes unlabeled data to improve regression performance

Operates directly within the data's intrinsic manifold structure

Abstract

We introduce a novel diffusion-based spectral algorithm to tackle regression analysis on high-dimensional data, particularly data embedded within lower-dimensional manifolds. Traditional spectral algorithms often fall short in such contexts, primarily due to the reliance on predetermined kernel functions, which inadequately address the complex structures inherent in manifold-based data. By employing graph Laplacian approximation, our method uses the local estimation property of heat kernel, offering an adaptive, data-driven approach to overcome this obstacle. Another distinct advantage of our algorithm lies in its semi-supervised learning framework, enabling it to fully use the additional unlabeled data. This ability enhances the performance by allowing the algorithm to dig the spectrum and curvature of the data manifold, providing a more comprehensive understanding of the dataset. Moreover, our algorithm performs in an entirely data-driven manner, operating directly within the intrinsic manifold structure of the data, without requiring any predefined manifold information. We provide a convergence analysis of our algorithm. Our findings reveal that the algorithm achieves a convergence rate that depends solely on the intrinsic dimension of the underlying manifold, thereby avoiding the curse of dimensionality associated with the higher ambient dimension.