Relational Deep Dive: Error-Aware Queries Over Unstructured Data

TL;DR

ReDD is a novel framework that dynamically discovers schemas and extracts data from unstructured sources with error guarantees, significantly reducing extraction errors and enabling accurate analytical queries.

Contribution

ReDD introduces a two-stage schema discovery and data population framework with error-aware guarantees, including the SCAPE method for calibrated error detection.

Findings

Reduces extraction errors from 30% to below 1%.

Achieves 100% schema recall and high precision.

Enables accuracy-cost trade-offs for high-stakes queries.

Abstract

Unstructured data is pervasive, but analytical queries demand structured representations, creating a significant extraction challenge. Existing methods like RAG lack schema awareness and struggle with cross-document alignment, leading to high error rates. We propose ReDD (Relational Deep Dive), a framework that dynamically discovers query-specific schemas, populates relational tables, and ensures error-aware extraction with provable guarantees. ReDD features a two-stage pipeline: (1) Iterative Schema Discovery (ISD) identifies minimal, joinable schemas tailored to each query, and (2) Tabular Data Population (TDP) extracts and corrects data using lightweight classifiers trained on LLM hidden states. A main contribution of ReDD is SCAPE, a statistically calibrated method for error detection with coverage guarantees, and SCAPE-HYB, a hybrid approach that optimizes the trade-off between accuracy and human correction costs. Experiments across diverse datasets demonstrate ReDD's effectiveness, reducing data extraction errors from up to 30% to below 1% while maintaining high schema completeness (100% recall) and precision. ReDD's modular design enables fine-grained control over accuracy-cost trade-offs, making it a robust solution for high-stakes analytical queries over unstructured corpora.