Reliable Answers for Recurring Questions: Boosting Text-to-SQL Accuracy with Template Constrained Decoding

TL;DR

TeCoD enhances Text-to-SQL accuracy and efficiency by using reusable templates and grammar-constrained decoding, significantly improving performance on complex schemas.

Contribution

Introduces Template Constrained Decoding (TeCoD), a novel system that leverages query templates and grammar constraints to improve Text-to-SQL generation.

Findings

Up to 36% higher execution accuracy than in-context learning.

2.2x lower latency on matched queries.

Effective template matching with a fine-tuned natural language inference model.

Abstract

Large language models (LLMs) have revolutionized Text-to-SQL generation, allowing users to query structured data using natural language with growing ease. Yet, real-world deployment remains challenging, especially in complex or unseen schemas, due to inconsistent accuracy and the risk of generating invalid SQL. We introduce Template Constrained Decoding (TeCoD), a system that addresses these limitations by harnessing the recurrence of query patterns in labeled workloads. TeCoD converts historical NL-SQL pairs into reusable templates and introduces a robust template selection module that uses a fine-tuned natural language inference model to match or reject queries efficiently. Once the template is selected, TeCoD enforces it during SQL generation through grammar-constrained decoding, implemented via a novel partitioned strategy that ensures both syntactic validity and efficiency. Together, these components yield up to 36% higher execution accuracy than in-context learning (ICL) and 2.2x lower latency on matched queries.