SEA-SQL: Semantic-Enhanced Text-to-SQL with Adaptive Refinement

TL;DR

SEA-SQL introduces a cost-efficient, adaptive refinement framework for Text-to-SQL tasks that enhances performance by semantic schema augmentation and dynamic execution verification, reducing reliance on expensive large models.

Contribution

The paper proposes SEA-SQL, a novel framework combining semantic schema enhancement, adaptive bias elimination, and dynamic execution adjustment for efficient Text-to-SQL conversion.

Findings

Achieves state-of-the-art results with GPT-3.5 at 9-58% of the original cost.

Comparable performance to GPT-4 with only 0.9-5.3% of the cost.

Effectively reduces resource expenditure while maintaining high accuracy.

Abstract

Recent advancements in large language models (LLMs) have significantly contributed to the progress of the Text-to-SQL task. A common requirement in many of these works is the post-correction of SQL queries. However, the majority of this process entails analyzing error cases to develop prompts with rules that eliminate model bias. And there is an absence of execution verification for SQL queries. In addition, the prevalent techniques primarily depend on GPT-4 and few-shot prompts, resulting in expensive costs. To investigate the effective methods for SQL refinement in a cost-efficient manner, we introduce Semantic-Enhanced Text-to-SQL with Adaptive Refinement (SEA-SQL), which includes Adaptive Bias Elimination and Dynamic Execution Adjustment, aims to improve performance while minimizing resource expenditure with zero-shot prompts. Specifically, SEA-SQL employs a semantic-enhanced schema to augment database information and optimize SQL queries. During the SQL query generation, a fine-tuned adaptive bias eliminator is applied to mitigate inherent biases caused by the LLM. The dynamic execution adjustment is utilized to guarantee the executability of the bias eliminated SQL query. We conduct experiments on the Spider and BIRD datasets to demonstrate the effectiveness of this framework. The results demonstrate that SEA-SQL achieves state-of-the-art performance in the GPT3.5 scenario with 9%-58% of the generation cost. Furthermore, SEA-SQL is comparable to GPT-4 with only 0.9%-5.3% of the generation cost.