PIPER: Content-Based Table Search via profiling and LLM-Generated Pseudoqueries

TL;DR

PIPER is a novel content-based table search method leveraging table profiles and LLM-generated pseudoqueries to improve dataset retrieval in environments with poor metadata quality.

Contribution

The paper introduces PIPER, a new retrieval approach that uses LLM-generated pseudoqueries and table profiling for effective dataset search without relying on metadata.

Findings

PIPER outperforms classical metadata-based baselines.

PIPER surpasses existing TableQA retrieval methods.

Content-driven retrieval improves dataset search in poor-metadata settings.

Abstract

The rapid growth of tabular datasets in data lakes, data spaces, and open data portals makes effective dataset search essential for reuse and analysis. Existing search systems rely mainly on metadata, which is often incomplete or low quality, especially for tables whose meaning depends on both schema and cell values. Recent advances in Large Language Models (LLMs) enable richer, content-based representations of tables. However, prior LLM-based retrieval methods have focused on Table Question Answering, where the goal is to select a single table to answer a question, rather than retrieve and rank relevant datasets. We propose PIPER, a content-driven retrieval method for tabular datasets that uses table profiles and LLM-generated queries embedded for dense retrieval. Designed for dataset search in poor-metadata settings, PIPER outperforms both classical metadata-based baselines and strong TableQA retrieval methods, demonstrating the value of LLM-based content modeling for tabular dataset search.