Turing: an Accurate and Interpretable Multi-Hypothesis Cross-Domain Natural Language Database Interface

TL;DR

Turing is a natural language database interface that uses a novel cross-domain semantic parser with value prediction to achieve high accuracy, complemented by an interactive explanation system for better user understanding.

Contribution

The paper introduces Turing, a new NLDB system with a novel value prediction method and an interactive explanation feature, improving cross-domain semantic parsing accuracy.

Findings

75.1% execution accuracy on Spider validation set

78.3% top-5 beam execution accuracy

Effective natural language explanations for SQL hypotheses

Abstract

A natural language database interface (NLDB) can democratize data-driven insights for non-technical users. However, existing Text-to-SQL semantic parsers cannot achieve high enough accuracy in the cross-database setting to allow good usability in practice. This work presents Turing, a NLDB system toward bridging this gap. The cross-domain semantic parser of Turing with our novel value prediction method achieves $75.1\%$ execution accuracy, and $78.3\%$ top-5 beam execution accuracy on the Spider validation set. To benefit from the higher beam accuracy, we design an interactive system where the SQL hypotheses in the beam are explained step-by-step in natural language, with their differences highlighted. The user can then compare and judge the hypotheses to select which one reflects their intention if any. The English explanations of SQL queries in Turing are produced by our high-precision natural language generation system based on synchronous grammars.