Fast-and-Frugal Text-Graph Transformers are Effective Link Predictors

TL;DR

The paper introduces FnF-TG Transformers, a fast and resource-efficient model that combines textual and structural data for improved inductive link prediction in knowledge graphs, outperforming previous methods.

Contribution

It presents a novel Transformer framework that encodes ego-graphs efficiently, enabling fully inductive learning with new datasets for unseen relations.

Findings

Achieves superior performance on three datasets.

Reduces reliance on resource-intensive textual encoders.

Extends inductive learning to relation descriptions.

Abstract

We propose Fast-and-Frugal Text-Graph (FnF-TG) Transformers, a Transformer-based framework that unifies textual and structural information for inductive link prediction in text-attributed knowledge graphs. We demonstrate that, by effectively encoding ego-graphs (1-hop neighbourhoods), we can reduce the reliance on resource-intensive textual encoders. This makes the model both fast at training and inference time, as well as frugal in terms of cost. We perform a comprehensive evaluation on three popular datasets and show that FnF-TG can achieve superior performance compared to previous state-of-the-art methods. We also extend inductive learning to a fully inductive setting, where relations don't rely on transductive (fixed) representations, as in previous work, but are a function of their textual description. Additionally, we introduce new variants of existing datasets, specifically designed to test the performance of models on unseen relations at inference time, thus offering a new test-bench for fully inductive link prediction.