Hyperboloid GPLVM for Discovering Continuous Hierarchies via Nonparametric Estimation

TL;DR

This paper introduces hyperboloid Gaussian process latent variable models (hGP-LVMs) that embed high-dimensional hierarchical data into low-dimensional hyperbolic space using nonparametric Bayesian methods.

Contribution

It proposes novel hGP-LVM variants with effective hierarchical embedding, incorporating Riemannian optimization and Bayesian inference techniques.

Findings

hGP-LVMs successfully represent high-dimensional hierarchies in low-dimensional spaces.

The models demonstrate effective hierarchical embedding on multiple datasets.

Bayesian variants improve the robustness of the hierarchical representations.

Abstract

Dimensionality reduction (DR) offers a useful representation of complex high-dimensional data. Recent DR methods focus on hyperbolic geometry to derive a faithful low-dimensional representation of hierarchical data. However, existing methods are based on neighbor embedding, frequently ruining the continual relation of the hierarchies. This paper presents hyperboloid Gaussian process (GP) latent variable models (hGP-LVMs) to embed high-dimensional hierarchical data with implicit continuity via nonparametric estimation. We adopt generative modeling using the GP, which brings effective hierarchical embedding and executes ill-posed hyperparameter tuning. This paper presents three variants that employ original point, sparse point, and Bayesian estimations. We establish their learning algorithms by incorporating the Riemannian optimization and active approximation scheme of GP-LVM. For Bayesian inference, we further introduce the reparameterization trick to realize Bayesian latent variable learning. In the last part of this paper, we apply hGP-LVMs to several datasets and show their ability to represent high-dimensional hierarchies in low-dimensional spaces.