SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data

TL;DR

SQLPrompt enhances few-shot Text-to-SQL performance by innovative prompt design and execution-based consistency decoding, significantly narrowing the gap with fine-tuning methods using minimal labeled data.

Contribution

The paper introduces SQLPrompt, a novel approach combining prompt design, execution-based consistency, and diversification techniques to improve few-shot Text-to-SQL with large language models.

Findings

SQLPrompt outperforms previous few-shot methods significantly.

It closes the performance gap with fine-tuning approaches.

Diversification strategies improve SQL proposal quality.

Abstract

Text-to-SQL aims to automate the process of generating SQL queries on a database from natural language text. In this work, we propose "SQLPrompt", tailored to improve the few-shot prompting capabilities of Text-to-SQL for Large Language Models (LLMs). Our methods include innovative prompt design, execution-based consistency decoding strategy which selects the SQL with the most consistent execution outcome among other SQL proposals, and a method that aims to improve performance by diversifying the SQL proposals during consistency selection with different prompt designs ("MixPrompt") and foundation models ("MixLLMs"). We show that \emph{SQLPrompt} outperforms previous approaches for in-context learning with few labeled data by a large margin, closing the gap with finetuning state-of-the-art with thousands of labeled data.