XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQL

TL;DR

XiYan-SQL is a new multi-generator framework that improves Text-to-SQL performance by generating diverse SQL candidates, filtering schemas, and selecting the best query, achieving state-of-the-art results.

Contribution

It introduces a multi-generator ensemble with a schema filter and candidate selection, enhancing SQL generation quality and robustness over previous methods.

Findings

Achieves 75.63% on BIRD benchmark, setting new SOTA.

Attains 89.65% accuracy on Spider test set, surpassing previous methods.

Demonstrates robustness and effectiveness through comprehensive analysis.

Abstract

To leverage the advantages of LLM in addressing challenges in the Text-to-SQL task, we present XiYan-SQL, an innovative framework effectively generating and utilizing multiple SQL candidates. It consists of three components: 1) a Schema Filter module filtering and obtaining multiple relevant schemas; 2) a multi-generator ensemble approach generating multiple highquality and diverse SQL queries; 3) a selection model with a candidate reorganization strategy implemented to obtain the optimal SQL query. Specifically, for the multi-generator ensemble, we employ a multi-task fine-tuning strategy to enhance the capabilities of SQL generation models for the intrinsic alignment between SQL and text, and construct multiple generation models with distinct generation styles by fine-tuning across different SQL formats. The experimental results and comprehensive analysis demonstrate the effectiveness and robustness of our framework. Overall, XiYan-SQL achieves a new SOTA performance of 75.63% on the notable BIRD benchmark, surpassing all previous methods. It also attains SOTA performance on the Spider test set with an accuracy of 89.65%.