Improving LLM-based Ontology Matching with fine-tuning on synthetic data

TL;DR

This paper presents a novel approach that uses synthetic data generation and fine-tuning of Large Language Models to improve their performance in ontology matching tasks, demonstrating significant gains on multiple datasets.

Contribution

It introduces a new method for automatically generating training data and fine-tuning LLMs specifically for ontology matching, enhancing zero-shot performance.

Findings

Fine-tuned LLMs outperform base models on ontology matching datasets.

Synthetic dataset generation effectively adapts LLMs for domain-specific tasks.

The approach improves matching accuracy across diverse ontology datasets.

Abstract

Large Language Models (LLMs) are increasingly being integrated into various components of Ontology Matching pipelines. This paper investigates the capability of LLMs to perform ontology matching directly on ontology modules and generate the corresponding alignments. Furthermore, it is explored how a dedicated fine-tuning strategy can enhance the model's matching performance in a zero-shot setting. The proposed method incorporates a search space reduction technique to select relevant subsets from both source and target ontologies, which are then used to automatically construct prompts. Recognizing the scarcity of reference alignments for training, a novel LLM-based approach is introduced for generating a synthetic dataset. This process creates a corpus of ontology submodule pairs and their corresponding reference alignments, specifically designed to fine-tune an LLM for the ontology matching task. The proposed approach was evaluated on the Conference, Geolink, Enslaved, Taxon, and Hydrography datasets from the OAEI complex track. The results demonstrate that the LLM fine-tuned on the synthetically generated data exhibits superior performance compared to the non-fine-tuned base model. The key contribution is a strategy that combines automatic dataset generation with fine-tuning to effectively adapt LLMs for ontology matching tasks.