Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks

TL;DR

This paper introduces a scalable Weibull graph attention autoencoder that combines hierarchical Bayesian models with Weibull-based inference to effectively model complex document networks, capturing hierarchical and semantic relationships.

Contribution

It develops a novel hierarchical Bayesian model, GPGBN, and integrates it with Weibull-based graph autoencoders to improve latent document representation quality.

Findings

Models extract high-quality hierarchical latent representations.

Achieve promising performance on graph analytic tasks.

Effectively model hierarchical and semantic document relationships.

Abstract

Although existing variational graph autoencoders (VGAEs) have been widely used for modeling and generating graph-structured data, most of them are still not flexible enough to approximate the sparse and skewed latent node representations, especially those of document relational networks (DRNs) with discrete observations. To analyze a collection of interconnected documents, a typical branch of Bayesian models, specifically relational topic models (RTMs), has proven their efficacy in describing both link structures and document contents of DRNs, which motives us to incorporate RTMs with existing VGAEs to alleviate their potential issues when modeling the generation of DRNs. In this paper, moving beyond the sophisticated approximate assumptions of traditional RTMs, we develop a graph Poisson factor analysis (GPFA), which provides analytic conditional posteriors to improve the inference accuracy, and extend GPFA to a multi-stochastic-layer version named graph Poisson gamma belief network (GPGBN) to capture the hierarchical document relationships at multiple semantic levels. Then, taking GPGBN as the decoder, we combine it with various Weibull-based graph inference networks, resulting in two variants of Weibull graph auto-encoder (WGAE), equipped with model inference algorithms. Experimental results demonstrate that our models can extract high-quality hierarchical latent document representations and achieve promising performance on various graph analytic tasks.