Grounded Adaptation for Zero-shot Executable Semantic Parsing

TL;DR

This paper introduces GAZP, a novel method for zero-shot executable semantic parsing that synthesizes and verifies data in new environments to improve parser adaptation, outperforming traditional data-augmentation techniques.

Contribution

GAZP combines a forward parser with a backward generator to synthesize and verify environment-specific data, enabling effective zero-shot adaptation for semantic parsing.

Findings

GAZP improves accuracy on Spider, Sparc, and CoSQL datasets.

Cycle-consistency verification is crucial for successful adaptation.

Performance increases with more GAZP-synthesized data.

Abstract

We propose Grounded Adaptation for Zero-shot Executable Semantic Parsing (GAZP) to adapt an existing semantic parser to new environments (e.g. new database schemas). GAZP combines a forward semantic parser with a backward utterance generator to synthesize data (e.g. utterances and SQL queries) in the new environment, then selects cycle-consistent examples to adapt the parser. Unlike data-augmentation, which typically synthesizes unverified examples in the training environment, GAZP synthesizes examples in the new environment whose input-output consistency are verified. On the Spider, Sparc, and CoSQL zero-shot semantic parsing tasks, GAZP improves logical form and execution accuracy of the baseline parser. Our analyses show that GAZP outperforms data-augmentation in the training environment, performance increases with the amount of GAZP-synthesized data, and cycle-consistency is central to successful adaptation.