Scalable Deep Generative Relational Models with High-Order Node Dependence

TL;DR

This paper introduces SDREM, a scalable deep generative model that captures high-order node dependencies in relational data, improving link prediction in large sparse networks.

Contribution

It incorporates high-order neighborhood information and a novel data augmentation for efficient Gibbs sampling, advancing scalable relational modeling.

Findings

Improved link prediction accuracy on real-world datasets.

Efficient Gibbs sampling due to novel data augmentation.

Scales well with large sparse networks.

Abstract

We propose a probabilistic framework for modelling and exploring the latent structure of relational data. Given feature information for the nodes in a network, the scalable deep generative relational model (SDREM) builds a deep network architecture that can approximate potential nonlinear mappings between nodes' feature information and the nodes' latent representations. Our contribution is two-fold: (1) We incorporate high-order neighbourhood structure information to generate the latent representations at each node, which vary smoothly over the network. (2) Due to the Dirichlet random variable structure of the latent representations, we introduce a novel data augmentation trick which permits efficient Gibbs sampling. The SDREM can be used for large sparse networks as its computational cost scales with the number of positive links. We demonstrate its competitive performance through improved link prediction performance on a range of real-world datasets.