GEMMA-SQL: A Novel Text-to-SQL Model Based on Large Language Models

TL;DR

GEMMA-SQL is an efficient, open-source text-to-SQL model based on Gemma 2B, achieving high accuracy through resource-efficient fine-tuning and advanced prompting strategies, suitable for low-cost hardware deployment.

Contribution

Introduces GEMMA-SQL, a lightweight, fine-tuned text-to-SQL model leveraging prompt strategies, outperforming state-of-the-art models on the SPIDER benchmark.

Findings

Achieves 66.8% Test-Suite accuracy and 63.3% Exact Set Match accuracy.

Outperforms baselines like IRNet, RYANSQL, and CodeXDavinci.

Demonstrates effective prompt design and instruction tuning enhance performance.

Abstract

Text-to-SQL systems enable users to interact with structured databases using natural language, eliminating the need for specialized programming knowledge. In this work, we introduce GEMMA-SQL, a lightweight and efficient text-to-SQL model built upon the open-source Gemma 2B architecture. Unlike many large language models (LLMs), GEMMA-SQL is fine-tuned in a resource-efficient, iterative manner and can be deployed on low-cost hardware. Leveraging the SPIDER benchmark for training and evaluation, GEMMA-SQL combines multiple prompting strategies, including few-shot learning, to enhance SQL query generation accuracy. The instruction-tuned variant, GEMMA-SQL Instruct, achieves 66.8% Test-Suite accuracy and 63.3% Exact Set Match accuracy, outperforming several state-of-the-art baselines such as IRNet, RYANSQL, and CodeXDavinci. The proposed approach demonstrates that effective prompt design and targeted instruction tuning can significantly boost performance while maintaining high scalability and adaptability. These results position GEMMA-SQL as a practical, open-source alternative for robust and accessible text-to-SQL systems.