SQLfuse: Enhancing Text-to-SQL Performance through Comprehensive LLM Synergy

TL;DR

SQLfuse is a system that significantly improves Text-to-SQL translation by integrating open-source LLMs with specialized modules for schema understanding, query generation, and iterative refinement, achieving top performance on benchmarks.

Contribution

The paper introduces SQLfuse, a novel framework that leverages open-source LLMs with multiple modules to enhance accuracy and usability in Text-to-SQL tasks, especially for complex queries.

Findings

Achieves leading results on the Spider Leaderboard.

Demonstrates practical deployment by Ant Group.

Enhances SQL query quality through iterative feedback.

Abstract

Text-to-SQL conversion is a critical innovation, simplifying the transition from complex SQL to intuitive natural language queries, especially significant given SQL's prevalence in the job market across various roles. The rise of Large Language Models (LLMs) like GPT-3.5 and GPT-4 has greatly advanced this field, offering improved natural language understanding and the ability to generate nuanced SQL statements. However, the potential of open-source LLMs in Text-to-SQL applications remains underexplored, with many frameworks failing to leverage their full capabilities, particularly in handling complex database queries and incorporating feedback for iterative refinement. Addressing these limitations, this paper introduces SQLfuse, a robust system integrating open-source LLMs with a suite of tools to enhance Text-to-SQL translation's accuracy and usability. SQLfuse features four modules: schema mining, schema linking, SQL generation, and a SQL critic module, to not only generate but also continuously enhance SQL query quality. Demonstrated by its leading performance on the Spider Leaderboard and deployment by Ant Group, SQLfuse showcases the practical merits of open-source LLMs in diverse business contexts.