Knowledge Graphs for Enhancing Large Language Models in Entity Disambiguation

TL;DR

This paper proposes using Knowledge Graphs to improve zero-shot entity disambiguation in Large Language Models by leveraging hierarchical entity representations and enriched prompts, resulting in better performance and adaptability.

Contribution

It introduces a novel method that integrates Knowledge Graphs with LLMs for entity disambiguation, enhancing accuracy without retraining the models.

Findings

Outperforms non-enhanced LLMs on ED datasets

Enriched prompts improve disambiguation accuracy

Higher adaptability compared to task-specific models

Abstract

Recent advances in Large Language Models (LLMs) have positioned them as a prominent solution for Natural Language Processing tasks. Notably, they can approach these problems in a zero or few-shot manner, thereby eliminating the need for training or fine-tuning task-specific models. However, LLMs face some challenges, including hallucination and the presence of outdated knowledge or missing information from specific domains in the training data. These problems cannot be easily solved by retraining the models with new data as it is a time-consuming and expensive process. To mitigate these issues, Knowledge Graphs (KGs) have been proposed as a structured external source of information to enrich LLMs. With this idea, in this work we use KGs to enhance LLMs for zero-shot Entity Disambiguation (ED). For that purpose, we leverage the hierarchical representation of the entities' classes in a KG to gradually prune the candidate space as well as the entities' descriptions to enrich the input prompt with additional factual knowledge. Our evaluation on popular ED datasets shows that the proposed method outperforms non-enhanced and description-only enhanced LLMs, and has a higher degree of adaptability than task-specific models. Furthermore, we conduct an error analysis and discuss the impact of the leveraged KG's semantic expressivity on the ED performance.