Text2VectorSQL: Towards a Unified Interface for Vector Search and SQL Queries

TL;DR

This paper introduces Text2VectorSQL, a unified natural language interface for querying both structured and unstructured data, supported by a new dataset, benchmark, and evaluation metrics to advance research in integrated data retrieval.

Contribution

It formalizes the Text2VectorSQL task, provides a scalable data synthesis pipeline, creates the first comprehensive benchmark, and proposes novel evaluation metrics for unified data querying.

Findings

Baseline models perform well but face recall degradation with combined SQL filters and vector search.

The integrated approach reveals challenges in maintaining high recall in unified queries.

The work establishes foundational tools and benchmarks for future research in unified data interfaces.

Abstract

The proliferation of unstructured data poses a fundamental challenge to traditional database interfaces. While Text-to-SQL has democratized access to structured data, it remains incapable of interpreting semantic or multi-modal queries. Concurrently, vector search has emerged as the de facto standard for querying unstructured data, but its integration with SQL-termed VectorSQL-still relies on manual query crafting and lacks standardized evaluation methodologies, creating a significant gap between its potential and practical application. To bridge this fundamental gap, we introduce and formalize Text2VectorSQL, a novel task to establish a unified natural language interface for seamlessly querying both structured and unstructured data. To catalyze research in this new domain, we present a comprehensive foundational ecosystem, including: (1) A scalable and robust pipeline for synthesizing high-quality Text-to-VectorSQL training data. (2) VectorSQLBench, the first large-scale, multi-faceted benchmark for this task, encompassing 12 distinct combinations across three database backends (SQLite, PostgreSQL, ClickHouse) and four data sources (BIRD, Spider, arXiv, Wikipedia). (3) Several novel evaluation metrics designed for more nuanced performance analysis. Extensive experiments not only confirm strong baseline performance with our trained models, but also reveal the recall degradation challenge: the integration of SQL filters with vector search can lead to more pronounced result omissions than in conventional filtered vector search. By defining the core task, delivering the essential data and evaluation infrastructure, and identifying key research challenges, our work lays the essential groundwork to build the next generation of unified and intelligent data interfaces. Our repository is available at https://github.com/OpenDCAI/Text2VectorSQL.