EPI-SQL: Enhancing Text-to-SQL Translation with Error-Prevention Instructions

TL;DR

EPI-SQL enhances Text-to-SQL translation by using task-specific, contextualized error-prevention instructions with LLMs, achieving high accuracy without requiring few-shot examples.

Contribution

The paper introduces a zero-shot method that incorporates contextualized error-prevention instructions into LLM prompts for improved Text-to-SQL performance.

Findings

Achieves 85.1% execution accuracy on Spider benchmark.

Rivals advanced few-shot methods without needing examples.

Demonstrates the effectiveness of task-specific instructions in LLMs.

Abstract

The conversion of natural language queries into SQL queries, known as Text-to-SQL, is a critical yet challenging task. This paper introduces EPI-SQL, a novel methodological framework leveraging Large Language Models (LLMs) to enhance the performance of Text-to-SQL tasks. EPI-SQL operates through a four-step process. Initially, the method involves gathering instances from the Spider dataset on which LLMs are prone to failure. These instances are then utilized to generate general error-prevention instructions (EPIs). Subsequently, LLMs craft contextualized EPIs tailored to the specific context of the current task. Finally, these context-specific EPIs are incorporated into the prompt used for SQL generation. EPI-SQL is distinguished in that it provides task-specific guidance, enabling the model to circumvent potential errors for the task at hand. Notably, the methodology rivals the performance of advanced few-shot methods despite being a zero-shot approach. An empirical assessment using the Spider benchmark reveals that EPI-SQL achieves an execution accuracy of 85.1\%, underscoring its effectiveness in generating accurate SQL queries through LLMs. The findings indicate a promising direction for future research, i.e. enhancing instructions with task-specific and contextualized rules, for boosting LLMs' performance in NLP tasks.