Robust Text-to-SQL Generation with Execution-Guided Decoding

TL;DR

This paper introduces execution-guided decoding for neural semantic parsing, significantly improving text-to-SQL translation accuracy by excluding faulty queries during generation, and achieves state-of-the-art results on multiple datasets.

Contribution

The paper presents a novel execution guidance mechanism that enhances neural text-to-SQL models by leveraging SQL semantics during decoding, applicable to various model architectures.

Findings

Universal improvement across models and datasets

Achieved 83.8% accuracy on WikiSQL

Effective on datasets with diverse query complexity

Abstract

We consider the problem of neural semantic parsing, which translates natural language questions into executable SQL queries. We introduce a new mechanism, execution guidance, to leverage the semantics of SQL. It detects and excludes faulty programs during the decoding procedure by conditioning on the execution of partially generated program. The mechanism can be used with any autoregressive generative model, which we demonstrate on four state-of-the-art recurrent or template-based semantic parsing models. We demonstrate that execution guidance universally improves model performance on various text-to-SQL datasets with different scales and query complexity: WikiSQL, ATIS, and GeoQuery. As a result, we achieve new state-of-the-art execution accuracy of 83.8% on WikiSQL.