Hyperbolic Graph Embedding with Enhanced Semi-Implicit Variational Inference

TL;DR

This paper introduces a novel hyperbolic graph embedding method using semi-implicit variational inference with mutual information regularization, improving the modeling of hierarchical and complex relational data.

Contribution

It combines hyperbolic geometry with semi-implicit variational inference and mutual information regularization to enhance graph representation learning.

Findings

Outperforms existing graph VAEs in link prediction and node classification

Effectively captures hierarchical and complex data structures

Enhances the quality of high-level graph representations

Abstract

Efficient modeling of relational data arising in physical, social, and information sciences is challenging due to complicated dependencies within the data. In this work, we build off of semi-implicit graph variational auto-encoders to capture higher-order statistics in a low-dimensional graph latent representation. We incorporate hyperbolic geometry in the latent space through a Poincare embedding to efficiently represent graphs exhibiting hierarchical structure. To address the naive posterior latent distribution assumptions in classical variational inference, we use semi-implicit hierarchical variational Bayes to implicitly capture posteriors of given graph data, which may exhibit heavy tails, multiple modes, skewness, and highly correlated latent structures. We show that the existing semi-implicit variational inference objective provably reduces information in the observed graph. Based on this observation, we estimate and add an additional mutual information term to the semi-implicit variational inference learning objective to capture rich correlations arising between the input and latent spaces. We show that the inclusion of this regularization term in conjunction with the Poincare embedding boosts the quality of learned high-level representations and enables more flexible and faithful graphical modeling. We experimentally demonstrate that our approach outperforms existing graph variational auto-encoders both in Euclidean and in hyperbolic spaces for edge link prediction and node classification.