Inverse Kernel Decomposition

TL;DR

Inverse Kernel Decomposition (IKD) is a novel nonlinear dimensionality reduction method that uses eigen-decomposition, offering a balance of performance and computational efficiency, especially in noisy data scenarios.

Contribution

The paper introduces IKD, a new eigen-decomposition-based nonlinear dimensionality reduction technique inspired by GPLVMs, with solutions for noisy data and competitive performance.

Findings

IKD outperforms other eigen-decomposition methods in experiments.

IKD achieves comparable results to optimization-based methods with faster speed.

Proposed solutions improve stability in noisy data conditions.

Abstract

The state-of-the-art dimensionality reduction approaches largely rely on complicated optimization procedures. On the other hand, closed-form approaches requiring merely eigen-decomposition do not have enough sophistication and nonlinearity. In this paper, we propose a novel nonlinear dimensionality reduction method -- Inverse Kernel Decomposition (IKD) -- based on an eigen-decomposition of the sample covariance matrix of data. The method is inspired by Gaussian process latent variable models (GPLVMs) and has comparable performance with GPLVMs. To deal with very noisy data with weak correlations, we propose two solutions -- blockwise and geodesic -- to make use of locally correlated data points and provide better and numerically more stable latent estimations. We use synthetic datasets and four real-world datasets to show that IKD is a better dimensionality reduction method than other eigen-decomposition-based methods, and achieves comparable performance against optimization-based methods with faster running speeds. Open-source IKD implementation in Python can be accessed at this \url{https://github.com/JerrySoybean/ikd}.