Retrieval, Reasoning, Re-ranking: A Context-Enriched Framework for Knowledge Graph Completion

TL;DR

This paper introduces KGR3, a novel framework that combines retrieval, reasoning, and re-ranking modules using large language models to enhance knowledge graph completion, significantly improving accuracy over existing methods.

Contribution

KGR3 is a new context-enriched framework integrating retrieval, reasoning, and re-ranking modules with LLMs for improved KGC performance.

Findings

Achieves up to 12.3% improvement in Hits@1 on FB15k237

Achieves up to 5.6% improvement in Hits@1 on WN18RR

Consistently outperforms baseline KGC methods across datasets.

Abstract

The Knowledge Graph Completion~(KGC) task aims to infer the missing entity from an incomplete triple. Existing embedding-based methods rely solely on triples in the KG, which is vulnerable to specious relation patterns and long-tail entities. On the other hand, text-based methods struggle with the semantic gap between KG triples and natural language. Apart from triples, entity contexts (e.g., labels, descriptions, aliases) also play a significant role in augmenting KGs. To address these limitations, we propose KGR3, a context-enriched framework for KGC. KGR3 is composed of three modules. Firstly, the Retrieval module gathers supporting triples from the KG, collects plausible candidate answers from a base embedding model, and retrieves context for each related entity. Then, the Reasoning module employs a large language model to generate potential answers for each query triple. Finally, the Re-ranking module combines candidate answers from the two modules mentioned above, and fine-tunes an LLM to provide the best answer. Extensive experiments on widely used datasets demonstrate that KGR3 consistently improves various KGC methods. Specifically, the best variant of KGR3 achieves absolute Hits@1 improvements of 12.3% and 5.6% on the FB15k237 and WN18RR datasets.