Gaussian Relational Graph Transformer

TL;DR

GelGT is a novel Gaussian relational graph transformer that effectively captures long-range dependencies and jointly models structural, semantic, and temporal information, leading to state-of-the-art results.

Contribution

The paper introduces GelGT, a Gaussian relational graph transformer with a structure-semantic sampling strategy and Gaussian attention, addressing limitations of existing models.

Findings

Achieves up to 13.8% improvement in predictive performance.

Effectively models long-range dependencies and temporal information.

Outperforms existing relational graph learning methods on real-world datasets.

Abstract

Relational graph learning models relational databases as graphs and has demonstrated superior performance on a wide range of relational predictive tasks. However, existing methods struggle to capture long-range dependencies due to information decay in their message-passing mechanisms, and recent relational graph transformers remain limited in jointly modeling structural, semantic, and temporal information. In this paper, we propose GelGT, a Gaussian relational graph transformer that explicitly addresses these challenges. GelGT introduces a structure-semantic collaborative sampling strategy to preserve structural connectivity while filtering irrelevant semantic information, and incorporates a Gaussian graph attention mechanism with a learnable Gaussian bias on the sampled subgraphs to dynamically encode temporal dependencies. Extensive experiments on various real-world datasets demonstrate that GelGT achieves state-of-the-art downstream task performance, with up to a 13.8% improvement in predictive performance.