Prompt-Matcher: Leveraging Large Models to Reduce Uncertainty in Schema Matching Results

TL;DR

This paper introduces Prompt-Matcher, a novel method using large language models to verify schema correspondences iteratively, reducing uncertainty and improving accuracy in schema matching tasks across diverse datasets.

Contribution

It proposes a new iterative approach leveraging GPT-4 with prompt templates for uncertainty reduction in schema matching, including a novel approximation algorithm for correspondence selection.

Findings

Outperforms brute-force algorithms in efficiency.

Achieves state-of-the-art verification accuracy with GPT-4.

Demonstrates robustness across benchmark datasets.

Abstract

Schema matching is the process of identifying correspondences between the elements of two given schemata, essential for database management systems, data integration, and data warehousing. For datasets across different scenarios, the optimal schema matching algorithm is different. For single algorithm, hyperparameter tuning also cases multiple results. All results assigned equal probabilities are stored in probabilistic databases to facilitate uncertainty management. The substantial degree of uncertainty diminishes the efficiency and reliability of data processing, thereby precluding the provision of more accurate information for decision-makers. To address this problem, we introduce a new approach based on fine-grained correspondence verification with specific prompt of Large Language Model. Our approach is an iterative loop that consists of three main components: (1) the correspondence selection algorithm, (2) correspondence verification, and (3) the update of probability distribution. The core idea is that correspondences intersect across multiple results, thereby linking the verification of correspondences to the reduction of uncertainty in candidate results. The task of selecting an optimal correspondence set to maximize the anticipated uncertainty reduction within a fixed budgetary framework is established as an NP-hard problem. We propose a novel $(1-1/e)$-approximation algorithm that significantly outperforms brute algorithm in terms of computational efficiency. To enhance correspondence verification, we have developed two prompt templates that enable GPT-4 to achieve state-of-the-art performance across two established benchmark datasets. Our comprehensive experimental evaluation demonstrates the superior effectiveness and robustness of the proposed approach.