PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agents

TL;DR

PV-SQL is a novel framework that enhances text-to-SQL systems by combining probing for contextual understanding and rule-based verification for constraint refinement, leading to improved accuracy and efficiency.

Contribution

The paper introduces PV-SQL, integrating probing and verification components to address deep understanding challenges in text-to-SQL tasks, outperforming existing baselines.

Findings

PV-SQL achieves 5% higher execution accuracy on BIRD benchmarks.

PV-SQL improves valid efficiency score by 20.8%.

PV-SQL uses fewer tokens than baseline methods.

Abstract

Text-to-SQL systems often struggle with deep contextual understanding, particularly for complex queries with subtle requirements. We present PV-SQL, an agentic framework that addresses these failures through two complementary components: Probe and Verify. The Probe component iteratively generates probing queries to retrieve concrete records from the database, resolving ambiguities in value formats, column semantics, and inter-table relationships to build richer contextual understanding. The Verify component employs a rule-based method to extract verifiable conditions and construct an executable checklist, enabling iterative SQL refinement that effectively reduces missing constraints. Experiments on the BIRD benchmarks show that PV-SQL outperforms the best text-to-SQL baseline by 5% in execution accuracy and 20.8% in valid efficiency score while consuming fewer tokens.