Lightweight Transformers for Zero-Shot and Fine-Tuned Text-to-SQL Generation Using Spider

TL;DR

This paper evaluates lightweight transformer models for zero-shot and fine-tuned text-to-SQL tasks on the Spider dataset, demonstrating that encoder-decoder models like T5-Small outperform others in low-resource settings.

Contribution

It introduces a reusable, model-agnostic pipeline for training lightweight transformers on text-to-SQL tasks, highlighting the effectiveness of T5-Small in schema-aware SQL generation.

Findings

Fine-tuned T5-Small achieves 27.8% LFAcc.

Encoder-decoder models outperform GPT-2 and BART-Small.

Pipeline supports future enhancements for schema linking.

Abstract

Text-to-SQL translation enables non-expert users to query relational databases using natural language, with applications in education and business intelligence. This study evaluates three lightweight transformer models - T5-Small, BART-Small, and GPT-2 - on the Spider dataset, focusing on low-resource settings. We developed a reusable, model-agnostic pipeline that tailors schema formatting to each model's architecture, training them across 1000 to 5000 iterations and evaluating on 1000 test samples using Logical Form Accuracy (LFAcc), BLEU, and Exact Match (EM) metrics. Fine-tuned T5-Small achieves the highest LFAcc (27.8%), outperforming BART-Small (23.98%) and GPT-2 (20.1%), highlighting encoder-decoder models' superiority in schema-aware SQL generation. Despite resource constraints limiting performance, our pipeline's modularity supports future enhancements, such as advanced schema linking or alternative base models. This work underscores the potential of compact transformers for accessible text-to-SQL solutions in resource-scarce environments.