Effectiveness of Prompt Optimization in NL2SQL Systems
Sairam Gurajada, Eser Kandogan, Sajjadur Rahman

TL;DR
This paper presents a prompt optimization framework for NL2SQL systems that enhances high-precision and high-performance SQL generation in production scenarios by carefully selecting exemplars and optimizing prompts.
Contribution
It introduces a novel prompt optimization approach that focuses on static exemplar selection tailored to production needs, improving NL2SQL accuracy and efficiency.
Findings
Preliminary results show improved SQL accuracy.
Optimized prompts reduce inference costs.
Framework enhances performance in production settings.
Abstract
NL2SQL approaches have greatly benefited from the impressive capabilities of large language models (LLMs). In particular, bootstrapping an NL2SQL system for a specific domain can be as simple as instructing an LLM with sufficient contextual information, such as schema details and translation demonstrations. However, building an accurate system still requires the rigorous task of selecting the right context for each query-including identifying relevant schema elements, cell values, and suitable exemplars that help the LLM understand domain-specific nuances. Retrieval-based methods have become the go-to approach for identifying such context. While effective, these methods introduce additional inference-time costs due to the retrieval process. In this paper, we argue that production scenarios demand high-precision, high-performance NL2SQL systems, rather than simply high-quality SQL…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsFocus · Sparse Evolutionary Training
