Entity Alignment with Noisy Annotations from Large Language Models

TL;DR

This paper introduces LLM4EA, a framework that leverages Large Language Models for entity alignment in knowledge graphs, effectively reducing annotation noise and space through active learning and label refinement.

Contribution

It proposes a novel active learning policy and an unsupervised label refiner to improve LLM-based entity alignment, addressing noise and efficiency issues.

Findings

Outperforms existing methods on four benchmark datasets.

Enhances robustness against noisy labels.

Reduces annotation effort significantly.

Abstract

Entity alignment (EA) aims to merge two knowledge graphs (KGs) by identifying equivalent entity pairs. While existing methods heavily rely on human-generated labels, it is prohibitively expensive to incorporate cross-domain experts for annotation in real-world scenarios. The advent of Large Language Models (LLMs) presents new avenues for automating EA with annotations, inspired by their comprehensive capability to process semantic information. However, it is nontrivial to directly apply LLMs for EA since the annotation space in real-world KGs is large. LLMs could also generate noisy labels that may mislead the alignment. To this end, we propose a unified framework, LLM4EA, to effectively leverage LLMs for EA. Specifically, we design a novel active learning policy to significantly reduce the annotation space by prioritizing the most valuable entities based on the entire inter-KG and intra-KG structure. Moreover, we introduce an unsupervised label refiner to continuously enhance label accuracy through in-depth probabilistic reasoning. We iteratively optimize the policy based on the feedback from a base EA model. Extensive experiments demonstrate the advantages of LLM4EA on four benchmark datasets in terms of effectiveness, robustness, and efficiency. Codes are available via https://github.com/chensyCN/llm4ea_official.