Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQL
Yeounoh Chung, Gaurav T. Kakkar, Yu Gan, Brenton Milne, Fatma Ozcan

TL;DR
This paper investigates how extended context windows in large language models can improve NL2SQL task performance by providing more contextual information, balancing accuracy gains with latency costs, and demonstrating robustness without fine-tuning.
Contribution
It is the first study to analyze the effects of extended context windows and additional contextual information on NL2SQL accuracy and latency using Google's Gemini-1.5-Pro model.
Findings
Long context LLMs are robust and do not lose relevant information.
Extended context improves NL2SQL accuracy without fine-tuning.
The approach achieves strong benchmark performance with manageable latency.
Abstract
Large Language Models (LLMs) have demonstrated impressive capabilities across a range of natural language processing tasks. In particular, improvements in reasoning abilities and the expansion of context windows have opened new avenues for leveraging these powerful models. NL2SQL is challenging in that the natural language question is inherently ambiguous, while the SQL generation requires a precise understanding of complex data schema and semantics. One approach to this semantic ambiguous problem is to provide more and sufficient contextual information. In this work, we explore the performance and the latency trade-offs of the extended context window (a.k.a., long context) offered by Google's state-of-the-art LLM (\textit{gemini-1.5-pro}). We study the impact of various contextual information, including column example values, question and SQL query pairs, user-provided hints, SQL…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Rights Management and Security
