Tokenization, Fusion and Decoupling: Bridging the Granularity Mismatch Between Large Language Models and Knowledge Graphs

TL;DR

This paper introduces KGT, a novel framework that bridges the granularity gap between LLMs and knowledge graphs by using dedicated entity tokens, fusion of features, and decoupled prediction, leading to improved KGC performance.

Contribution

KGT is the first to combine dedicated entity tokens, feature fusion, and decoupled prediction for effective knowledge graph completion with LLMs.

Findings

KGT outperforms existing methods on multiple benchmarks.

Specialized tokenization improves entity representation.

Fusion and decoupling enhance reasoning capabilities.

Abstract

Leveraging Large Language Models (LLMs) for Knowledge Graph Completion (KGC) is promising but hindered by a fundamental granularity mismatch. LLMs operate on fragmented token sequences, whereas entities are the fundamental units in knowledge graphs (KGs) scenarios. Existing approaches typically constrain predictions to limited candidate sets or align entities with the LLM's vocabulary by pooling multiple tokens or decomposing entities into fixed-length token sequences, which fail to capture both the semantic meaning of the text and the structural integrity of the graph. To address this, we propose KGT, a novel framework that uses dedicated entity tokens to enable efficient, full-space prediction. Specifically, we first introduce specialized tokenization to construct feature representations at the level of dedicated entity tokens. We then fuse pre-trained structural and textual features into these unified embeddings via a relation-guided gating mechanism, avoiding training from scratch. Finally, we implement decoupled prediction by leveraging independent heads to separate and combine semantic and structural reasoning. Experimental results show that KGT consistently outperforms state-of-the-art methods across multiple benchmarks.