Selective Demonstrations for Cross-domain Text-to-SQL
Shuaichen Chang, Eric Fosler-Lussier

TL;DR
This paper introduces ODIS, a demonstration selection framework that combines out-of-domain and synthetic in-domain examples to improve cross-domain text-to-SQL performance of large language models, outperforming existing methods.
Contribution
The paper proposes a novel demonstration selection method, ODIS, which effectively leverages hybrid data sources to enhance LLMs' cross-domain text-to-SQL capabilities without in-domain annotations.
Findings
ODIS outperforms baseline methods on two datasets.
ODIS improves execution accuracy by 1.1 and 11.8 points.
Hybrid demonstration retrieval benefits cross-domain generalization.
Abstract
Large language models (LLMs) with in-context learning have demonstrated impressive generalization capabilities in the cross-domain text-to-SQL task, without the use of in-domain annotations. However, incorporating in-domain demonstration examples has been found to greatly enhance LLMs' performance. In this paper, we delve into the key factors within in-domain examples that contribute to the improvement and explore whether we can harness these benefits without relying on in-domain annotations. Based on our findings, we propose a demonstration selection framework ODIS which utilizes both out-of-domain examples and synthetically generated in-domain examples to construct demonstrations. By retrieving demonstrations from hybrid sources, ODIS leverages the advantages of both, showcasing its effectiveness compared to baseline methods that rely on a single data source. Furthermore, ODIS…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Web Data Mining and Analysis
