PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL

TL;DR

This paper introduces PTD-SQL, a partitioning and targeted drilling approach that enhances LLMs' reasoning in Text-to-SQL tasks by focusing on specific problem categories, leading to state-of-the-art results.

Contribution

The study proposes query group partitioning and targeted drilling techniques to improve LLM reasoning in Text-to-SQL, demonstrating significant performance gains on benchmark datasets.

Findings

LLMs with PTD-SQL outperform or match SOTA on Spider and BIRD datasets.

Targeted drilling significantly improves models at their capability boundaries.

Models with different initial performances all benefit from the approach.

Abstract

Large Language Models (LLMs) have emerged as powerful tools for Text-to-SQL tasks, exhibiting remarkable reasoning capabilities. Different from tasks such as math word problems and commonsense reasoning, SQL solutions have a relatively fixed pattern. This facilitates the investigation of whether LLMs can benefit from categorical thinking, mirroring how humans acquire knowledge through inductive reasoning based on comparable examples. In this study, we propose that employing query group partitioning allows LLMs to focus on learning the thought processes specific to a single problem type, consequently enhancing their reasoning abilities across diverse difficulty levels and problem categories. Our experiments reveal that multiple advanced LLMs, when equipped with PTD-SQL, can either surpass or match previous state-of-the-art (SOTA) methods on the Spider and BIRD datasets. Intriguingly, models with varying initial performances have exhibited significant improvements, mainly at the boundary of their capabilities after targeted drilling, suggesting a parallel with human progress. Code is available at https://github.com/lrlbbzl/PTD-SQL.