BDIViz in Action: Interactive Curation and Benchmarking for Schema Matching Methods

TL;DR

BDIViz is an interactive visualization system that facilitates schema matching, validation, and benchmarking with human-in-the-loop capabilities, leveraging LLM assistance for data integration tasks.

Contribution

The paper introduces an extension to BDIViz enabling human-in-the-loop benchmarking, custom matcher integration, and real-time performance evaluation for schema matching methods.

Findings

Supports human validation of schema matches in an interactive heatmap.

Enables real-time benchmarking and comparison of different matching algorithms.

Facilitates high-quality ground-truth dataset creation through expert validation.

Abstract

Schema matching remains fundamental to data integration, yet evaluating and comparing matching methods is hindered by limited benchmark diversity and lack of interactive validation frameworks. BDIViz, recently published at IEEE VIS 2025, is an interactive visualization system for schema matching with LLM-assisted validation. Given source and target datasets, BDIViz applies automatic matching methods and visualizes candidates in an interactive heatmap with hierarchical navigation, zoom, and filtering. Users validate matches directly in the heatmap and inspect ambiguous cases using coordinated views that show attribute descriptions, example values, and distributions. An LLM assistant generates structured explanations for selected candidates to support decision-making. This demonstration showcases a new extension to BDIViz that addresses a critical need in data integration research: human-in-the-loop benchmarking and iterative matcher development. New matchers can be integrated through a standardized interface, while user validations become evolving ground truth for real-time performance evaluation. This enables benchmarking new algorithms, constructing high-quality ground-truth datasets through expert validation, and comparing matcher behavior across diverse schemas and domains. We demonstrate two complementary scenarios: (i) data harmonization, where users map a large tabular dataset to a target schema with value-level inspection and LLM-generated explanations; and (ii) developer-in-the-loop benchmarking, where developers integrate custom matchers, observe performance metrics, and refine their algorithms.