Generalized Graph Transformer Variational Autoencoder

TL;DR

This paper introduces GGT-VAE, a novel graph autoencoder that uses transformer-based self-attention and positional encoding to improve link prediction without message passing, showing superior results on benchmarks.

Contribution

The paper presents the first integration of generalized graph transformers with variational autoencoders for graph structure generation and link prediction.

Findings

GGT-VAE outperforms baseline models in ROC-AUC and Average Precision.

The model effectively captures structural patterns without message passing.

Experimental results validate the approach on multiple benchmark datasets.

Abstract

Graph link prediction has long been a central problem in graph representation learning in both network analysis and generative modeling. Recent progress in deep learning has introduced increasingly sophisticated architectures for capturing relational dependencies within graph-structured data. In this work, we propose the Generalized Graph Transformer Variational Autoencoder (GGT-VAE). Our model integrates Generalized Graph Transformer Architecture with Variational Autoencoder framework for link prediction. Unlike prior GraphVAE, GCN, or GNN approaches, GGT-VAE leverages transformer style global self-attention mechanism along with laplacian positional encoding to model structural patterns across nodes into a latent space without relying on message passing. Experimental results on several benchmark datasets demonstrate that GGT-VAE consistently achieves above-baseline performance in terms of ROC-AUC and Average Precision. To the best of our knowledge, this is among the first studies to explore graph structure generation using a generalized graph transformer backbone in a variational framework.