Adaptive Neighboring Selection Algorithm Based on Curvature Prediction in Manifold Learning

TL;DR

This paper introduces an adaptive neighbor selection algorithm based on curvature prediction for manifold learning, improving the selection of neighbors and enhancing embedding quality in dimensionality reduction tasks.

Contribution

It proposes a curvature-based adaptive neighbor selection method applicable to LLE and ISOMAP, addressing the lack of a generally accepted neighbor selection algorithm.

Findings

Residual variance reduced by 45.45% with the new method

Improved visualization quality in manifold embedding

Algorithm successfully finds optimal neighbor count K

Abstract

Recently manifold learning algorithm for dimensionality reduction attracts more and more interests, and various linear and nonlinear, global and local algorithms are proposed. The key step of manifold learning algorithm is the neighboring region selection. However, so far for the references we know, few of which propose a generally accepted algorithm to well select the neighboring region. So in this paper, we propose an adaptive neighboring selection algorithm, which successfully applies the LLE and ISOMAP algorithms in the test. It is an algorithm that can find the optimal K nearest neighbors of the data points on the manifold. And the theoretical basis of the algorithm is the approximated curvature of the data point on the manifold. Based on Riemann Geometry, Jacob matrix is a proper mathematical concept to predict the approximated curvature. By verifying the proposed algorithm on embedding Swiss roll from R3 to R2 based on LLE and ISOMAP algorithm, the simulation results show that the proposed adaptive neighboring selection algorithm is feasible and able to find the optimal value of K, making the residual variance relatively small and better visualization of the results. By quantitative analysis, the embedding quality measured by residual variance is increased 45.45% after using the proposed algorithm in LLE.