Large Language Models as Oracles for Ontology Alignment

TL;DR

This paper explores using Large Language Models as validation tools to improve ontology alignment accuracy, achieving top-tier results in a major evaluation initiative.

Contribution

It introduces a novel approach of employing LLMs as oracles for validation in ontology alignment, reducing human effort and enhancing performance.

Findings

LLMs achieved top-2 rank in OAEI 2025 bio-ml track.

Using LLMs as validation improves alignment quality.

The approach reduces human involvement in large ontology mappings.

Abstract

There are many methods and systems to tackle the ontology alignment problem, yet a major challenge persists in producing high-quality mappings among a set of input ontologies. Adopting a human-in-the-loop approach during the alignment process has become essential in applications requiring very accurate mappings. However, user involvement is expensive when dealing with large ontologies. In this paper, we analyse the feasibility of using Large Language Models (LLM) to aid the ontology alignment problem. LLMs are used only in the validation of a subset of correspondences for which there is high uncertainty. We have conducted an extensive analysis over several tasks of the Ontology Alignment Evaluation Initiative (OAEI), reporting in this paper the performance of several state-of-the-art LLMs using different prompt templates. Using LLMs as Oracles resulted in strong performance in the OAEI 2025, achieving the top-2 overall rank in the bio-ml track.