Towards Generalizable and Robust Text-to-SQL Parsing

TL;DR

This paper introduces a multi-stage TKK framework for text-to-SQL parsing that enhances generalization and robustness, achieving state-of-the-art results across multiple challenging datasets.

Contribution

The paper proposes a novel staged learning framework, TKK, to improve the generalizability and robustness of text-to-SQL parsers, addressing multiple challenges comprehensively.

Findings

State-of-the-art performance on Spider, SParC, and CoSQL datasets.

Effective in various generalization and robustness scenarios.

Improves SQL knowledge acquisition over spurious pattern learning.

Abstract

Text-to-SQL parsing tackles the problem of mapping natural language questions to executable SQL queries. In practice, text-to-SQL parsers often encounter various challenging scenarios, requiring them to be generalizable and robust. While most existing work addresses a particular generalization or robustness challenge, we aim to study it in a more comprehensive manner. In specific, we believe that text-to-SQL parsers should be (1) generalizable at three levels of generalization, namely i.i.d., zero-shot, and compositional, and (2) robust against input perturbations. To enhance these capabilities of the parser, we propose a novel TKK framework consisting of Task decomposition, Knowledge acquisition, and Knowledge composition to learn text-to-SQL parsing in stages. By dividing the learning process into multiple stages, our framework improves the parser's ability to acquire general SQL knowledge instead of capturing spurious patterns, making it more generalizable and robust. Experimental results under various generalization and robustness settings show that our framework is effective in all scenarios and achieves state-of-the-art performance on the Spider, SParC, and CoSQL datasets. Code can be found at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/tkk.