Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker

TL;DR

This paper introduces Bertrand-DR, a discriminative re-ranker that enhances generative text-to-SQL models by selecting the best query from beam outputs, significantly improving performance on complex queries.

Contribution

The paper presents a novel schema-agnostic BERT-based re-ranker that boosts generative text-to-SQL models by effectively selecting optimal queries from candidate lists.

Findings

Achieved top 4 score on the Spider leaderboard.

Improved performance when the correct query is in the candidate list.

Effective combination of generative models and re-rankers enhances accuracy.

Abstract

To access data stored in relational databases, users need to understand the database schema and write a query using a query language such as SQL. To simplify this task, text-to-SQL models attempt to translate a user's natural language question to corresponding SQL query. Recently, several generative text-to-SQL models have been developed. We propose a novel discriminative re-ranker to improve the performance of generative text-to-SQL models by extracting the best SQL query from the beam output predicted by the text-to-SQL generator, resulting in improved performance in the cases where the best query was in the candidate list, but not at the top of the list. We build the re-ranker as a schema agnostic BERT fine-tuned classifier. We analyze relative strengths of the text-to-SQL and re-ranker models across different query hardness levels, and suggest how to combine the two models for optimal performance. We demonstrate the effectiveness of the re-ranker by applying it to two state-of-the-art text-to-SQL models, and achieve top 4 score on the Spider leaderboard at the time of writing this article.