DPDisc: From Factoid Questions to Data Product Requests for Open-World Data Product Discovery over Tables and Text

TL;DR

DPDisc introduces the first large-scale benchmark for discovering coherent data products from hybrid table-text corpora, enabling automated retrieval of related data assets for business use cases.

Contribution

It presents DPDisc, a novel benchmark and DPForge pipeline for systematic data product discovery over tables and text, filling a key gap in existing datasets.

Findings

Baseline retrieval methods show performance gaps across domains.

DPDisc enables evaluation of structure-aware data discovery methods.

The benchmark facilitates future research in automated data product retrieval.

Abstract

Data products are reusable, self-contained assets designed for specific business use cases. Automating their discovery is of great industry interest, as it enables efficient data access in large data lakes and supports analytical workflows. However, no benchmark currently exists for data product discovery over hybrid table-text corpora. Existing datasets focus on answering single factoid questions over individual tables rather than assembling multiple related data assets into coherent products. To address this gap, we present DPDisc, the first large-scale benchmark for data product discovery, where systems must retrieve coherent collections of tables and passages to satisfy high-level Data Product Requests (DPRs). We introduce DPForge, an automated pipeline that systematically repurposes table-text QA datasets by clustering related tables and passages into coherent data products, generating professional-level analytical requests using an LLM ensemble, and validating quality through multi-phase LLM evaluation. DPDisc comprises 13,076 validated instances with full provenance, derived from three representative datasets spanning open-domain and financial domains. Baseline experiments with sparse, dense, and hybrid retrieval methods imply evaluation feasibility while revealing substantial performance gaps across domains, indicating opportunities for future research in structure-aware data product discovery. Code and datasets are available at: Dataset: https://huggingface.co/datasets/ibm-research/data-product-benchmark Code: https://github.com/ibm/data-product-benchmark