Rationalization Models for Text-to-SQL

TL;DR

This paper presents a framework for generating Chain-of-Thought rationales to improve text-to-SQL models, leading to better accuracy and explainability, especially for complex queries.

Contribution

It introduces a novel rationalization approach using iterative knowledge distillation and synthetic annotations to enhance text-to-SQL model training.

Findings

Step-by-step query generation improves execution accuracy.

Rationales enhance explainability of models.

Performance gains are notable for complex queries.

Abstract

We introduce a framework for generating Chain-of-Thought (CoT) rationales to enhance text-to-SQL model fine-tuning. These rationales consist of intermediate SQL statements and explanations, serving as incremental steps toward constructing the final SQL query. The process begins with manually annotating a small set of examples, which are then used to prompt a large language model in an iterative, dynamic few-shot knowledge distillation procedure from a teacher model. A rationalization model is subsequently trained on the validated decomposed queries, enabling extensive synthetic CoT annotations for text-to-SQL datasets. To evaluate the approach, we fine-tune small language models with and without these rationales on the BIRD dataset. Results indicate that step-by-step query generation improves execution accuracy, especially for moderately and highly complex queries, while also enhancing explainability.