Beyond Query-Level Comparison: Fine-Grained Reinforcement Learning for Text-to-SQL with Automated Interpretable Critiques

TL;DR

This paper introduces RuCo-C, a novel reinforcement learning framework that uses automated, interpretable critiques and query-specific evaluation rubrics to improve text-to-SQL models beyond coarse binary rewards.

Contribution

It presents a new generative judge model that provides fine-grained, interpretable feedback and a progressive exploration strategy for RL training in text-to-SQL tasks.

Findings

RuCo-C outperforms existing evaluation methods in text-to-SQL.

The framework achieves significant performance improvements.

Automated, interpretable critiques enhance model training effectiveness.

Abstract

Text-to-SQL, a pivotal natural language processing (NLP) task that converts textual queries into executable SQL, has seen substantial progress in recent years. However, existing evaluation and reward mechanisms used to train and assess the text-to-SQL models remain a critical bottleneck. Current approaches heavily rely on manually annotated gold SQL queries, which are costly to produce and impractical for large-scale evaluation. More importantly, most reinforcement learning (RL) methods in text-to-SQL leverage only the final binary execution outcome as the reward signal, a coarse-grained supervision that overlooks detailed structural and semantic errors from the perspective of rubrics. To address these challenges, we propose RuCo-C, a novel generative judge model for fine-grained, query-specific automatic evaluation using interpretable critiques without human intervention. Our framework first automatically generates query-specific evaluation rubrics for human-free annotation, linking them to interpretable critiques. Subsequently, it integrates densified reward feedback through a "progressive exploration" strategy during the RL training process, which dynamically adjusts the rewards to enhance the model's performance. Comprehensive experiments demonstrate that RuCo-C outperforms existing methods in text-to-SQL evaluation, yielding significant performance gains.