CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions

TL;DR

CoE-SQL enhances multi-turn text-to-SQL tasks by prompting LLMs with an edition chain of previous SQL queries, achieving state-of-the-art results on benchmarks without fine-tuning.

Contribution

Introduces CoE-SQL, a novel prompt design using an edition chain to improve LLM reasoning in multi-turn text-to-SQL tasks.

Findings

Outperforms existing in-context learning baselines

Achieves state-of-the-art results on SParC and CoSQL benchmarks

Competitive with fine-tuned models

Abstract

Recently, Large Language Models (LLMs) have been demonstrated to possess impressive capabilities in a variety of domains and tasks. We investigate the issue of prompt design in the multi-turn text-to-SQL task and attempt to enhance the LLMs' reasoning capacity when generating SQL queries. In the conversational context, the current SQL query can be modified from the preceding SQL query with only a few operations due to the context dependency. We introduce our method called CoE-SQL which can prompt LLMs to generate the SQL query based on the previously generated SQL query with an edition chain. We also conduct extensive ablation studies to determine the optimal configuration of our approach. Our approach outperforms different in-context learning baselines stably and achieves state-of-the-art performances on two benchmarks SParC and CoSQL using LLMs, which is also competitive to the SOTA fine-tuned models.