IncSQL: Training Incremental Text-to-SQL Parsers with Non-Deterministic Oracles

TL;DR

This paper introduces IncSQL, a sequence-to-action model for text-to-SQL parsing trained with non-deterministic oracles, achieving state-of-the-art accuracy on WikiSQL by accounting for multiple correct SQL queries.

Contribution

It proposes a novel training method using non-deterministic oracles for incremental SQL parsing, improving accuracy over traditional static oracle methods.

Findings

Achieved 83.7% execution accuracy on WikiSQL test set.

Improved accuracy by 2.1% over static oracle training.

Set a new state-of-the-art at 87.1% with execution-guided decoding.

Abstract

We present a sequence-to-action parsing approach for the natural language to SQL task that incrementally fills the slots of a SQL query with feasible actions from a pre-defined inventory. To account for the fact that typically there are multiple correct SQL queries with the same or very similar semantics, we draw inspiration from syntactic parsing techniques and propose to train our sequence-to-action models with non-deterministic oracles. We evaluate our models on the WikiSQL dataset and achieve an execution accuracy of 83.7% on the test set, a 2.1% absolute improvement over the models trained with traditional static oracles assuming a single correct target SQL query. When further combined with the execution-guided decoding strategy, our model sets a new state-of-the-art performance at an execution accuracy of 87.1%.