DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph

TL;DR

This paper introduces DCG-SQL, a method that uses a deep contextual schema link graph to improve demonstration retrieval and SQL generation in in-context learning for Text-to-SQL tasks, benefiting both large and small LLMs.

Contribution

It proposes a novel Deep Contextual Schema Link Graph for better demonstration retrieval, significantly enhancing Text-to-SQL performance across various LLM sizes.

Findings

Improved SQL accuracy on the Spider benchmark.

Enhanced performance with both hyper-scaled and small LLMs.

Demonstrated efficiency gains in demonstration retrieval.

Abstract

Text-to-SQL, which translates a natural language question into an SQL query, has advanced with in-context learning of Large Language Models (LLMs). However, existing methods show little improvement in performance compared to randomly chosen demonstrations, and significant performance drops when smaller LLMs (e.g., Llama 3.1-8B) are used. This indicates that these methods heavily rely on the intrinsic capabilities of hyper-scaled LLMs, rather than effectively retrieving useful demonstrations. In this paper, we propose a novel approach for effectively retrieving demonstrations and generating SQL queries. We construct a Deep Contextual Schema Link Graph, which contains key information and semantic relationship between a question and its database schema items. This graph-based structure enables effective representation of Text-to-SQL samples and retrieval of useful demonstrations for in-context learning. Experimental results on the Spider benchmark demonstrate the effectiveness of our approach, showing consistent improvements in SQL generation performance and efficiency across both hyper-scaled LLMs and small LLMs. The code is available at https://github.com/jjklle/DCG-SQL}{https://github.com/jjklle/DCG-SQL.