Every Step Counts: Step-Level Credit Assignment for Tool-Integrated Text-to-SQL

TL;DR

This paper introduces FineStep, a step-level credit assignment framework for tool-augmented Text-to-SQL, improving efficiency and performance by addressing reward sparsity and credit assignment issues.

Contribution

The paper proposes a novel step-level credit assignment mechanism and reward design for tool-integrated Text-to-SQL, achieving state-of-the-art results.

Findings

FineStep outperforms previous methods on BIRD benchmarks.

Reduces redundant tool interactions by 3.25% average EX gain.

Enhances reasoning efficiency and generalization in Text-to-SQL.

Abstract

Tool-integrated Text-to-SQL parsing has emerged as a promising paradigm, framing SQL generation as a sequential decision-making process interleaved with tool execution. However, existing reinforcement learning approaches mainly rely on coarse-grained outcome supervision, resulting in a fundamental credit assignment problem: models receive the same reward for any trajectory that yields the correct answer, even when intermediate steps are redundant, inefficient, or erroneous. Consequently, models are encouraged to explore suboptimal reasoning spaces, limiting both efficiency and generalization. To address this problem, we propose FineStep, a novel framework for step-level credit assignment in tool-augmented Text-to-SQL. First, we introduce a reward design with independent process rewards to alleviate the signal sparsity of outcome supervision. Next, we present a step-level credit assignment mechanism to precisely quantify the value of each reasoning step. Finally, we develop a policy optimization method based on step-level advantages for efficient updates. Extensive experiments on BIRD benchmarks show that FineStep achieves state-of-the-art performance and reduces redundant tool interactions, with a 3.25% average EX gain over GRPO at the 4B scale.