Filter-then-Generate: Large Language Models with Structure-Text Adapter for Knowledge Graph Completion

TL;DR

This paper introduces FtG, a novel method that enhances large language models for knowledge graph completion by combining a filter-then-generate approach with structure-text adapters, significantly improving performance over existing methods.

Contribution

The paper proposes a new instruction-tuning method with a filter-then-generate paradigm and a structure-text adapter to better utilize LLMs for knowledge graph completion.

Findings

FtG outperforms state-of-the-art methods in KGC tasks.

The filter-then-generate approach reduces hallucination issues.

Structured prompts improve the integration of graph information.

Abstract

Large Language Models (LLMs) present massive inherent knowledge and superior semantic comprehension capability, which have revolutionized various tasks in natural language processing. Despite their success, a critical gap remains in enabling LLMs to perform knowledge graph completion (KGC). Empirical evidence suggests that LLMs consistently perform worse than conventional KGC approaches, even through sophisticated prompt design or tailored instruction-tuning. Fundamentally, applying LLMs on KGC introduces several critical challenges, including a vast set of entity candidates, hallucination issue of LLMs, and under-exploitation of the graph structure. To address these challenges, we propose a novel instruction-tuning-based method, namely FtG. Specifically, we present a filter-then-generate paradigm and formulate the KGC task into a multiple-choice question format. In this way, we can harness the capability of LLMs while mitigating the issue casused by hallucinations. Moreover, we devise a flexible ego-graph serialization prompt and employ a structure-text adapter to couple structure and text information in a contextualized manner. Experimental results demonstrate that FtG achieves substantial performance gain compared to existing state-of-the-art methods. The instruction dataset and code are available at https://github.com/LB0828/FtG.