Leveraging Graph Structure in Seq2Seq Models for Knowledge Graph Link Prediction

TL;DR

This paper presents GA-S2S, a framework combining Seq2Seq models with relational graph attention to enhance knowledge graph link prediction by utilizing full subgraph structures.

Contribution

It introduces a novel integration of textual and graph features in Seq2Seq models for improved link prediction in knowledge graphs.

Findings

GA-S2S outperforms baseline models on CoDEx dataset.

Achieves up to 19% relative gain in link prediction accuracy.

Effectively captures multi-hop relational patterns.

Abstract

We introduce Graph-Augmented Sequence-to-Sequence (GA-S2S), a novel framework that integrates a T5-small encoder-decoder with a Relational Graph Attention Network (RGAT) to improve link prediction in knowledge graphs. While existing Seq2Seq models rely solely on surface-level textual descriptions of entities and relations and at best, flatten the neighborhoods of a query entity into a single linear sequence, thereby discarding the inherent graph structure, GA-S2S jointly encodes both textual features and the full $k$-hop subgraph topology surrounding the query entity. By integrating raw encoder outputs with RGAT's relation-aware embeddings, our model captures and leverages richer multi-hop relational patterns and textual information. Our preliminary experiments on the CoDEx dataset demonstrate that GA-S2S outperforms competitive Seq2Seq-based baseline models, achieving up to a 19\% relative gain in link prediction accuracy.