GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion

TL;DR

This paper introduces GLTW, a novel method that combines an improved Graph Transformer with Large Language Models to enhance Knowledge Graph Completion by effectively encoding structural information and improving prediction accuracy.

Contribution

The paper presents a new approach that integrates an improved Graph Transformer with LLMs for better structural encoding and deterministic predictions in KGC tasks.

Findings

GLTW outperforms state-of-the-art methods on multiple KG datasets.

The improved Graph Transformer effectively encodes local and global structural information.

The subgraph-based training boosts learning efficiency and prediction accuracy.

Abstract

Knowledge Graph Completion (KGC), which aims to infer missing or incomplete facts, is a crucial task for KGs. However, integrating the vital structural information of KGs into Large Language Models (LLMs) and outputting predictions deterministically remains challenging. To address this, we propose a new method called GLTW, which encodes the structural information of KGs and merges it with LLMs to enhance KGC performance. Specifically, we introduce an improved Graph Transformer (iGT) that effectively encodes subgraphs with both local and global structural information and inherits the characteristics of language model, bypassing training from scratch. Also, we develop a subgraph-based multi-classification training objective, using all entities within KG as classification objects, to boost learning efficiency.Importantly, we combine iGT with an LLM that takes KG language prompts as input.Our extensive experiments on various KG datasets show that GLTW achieves significant performance gains compared to SOTA baselines.