"What Do You Mean by That?" A Parser-Independent Interactive Approach for Enhancing Text-to-SQL

TL;DR

This paper introduces a parser-independent interactive method that involves users in the process to improve text-to-SQL translation accuracy across different datasets and parsers.

Contribution

It proposes a novel human-in-the-loop approach that enhances text-to-SQL systems without relying on specific parsers, improving performance with minimal user interaction.

Findings

PIIA improves text-to-SQL accuracy on WikiSQL and Spider datasets.

The approach works effectively with multiple state-of-the-art parsers.

Limited user interaction significantly boosts system performance.

Abstract

In Natural Language Interfaces to Databases systems, the text-to-SQL technique allows users to query databases by using natural language questions. Though significant progress in this area has been made recently, most parsers may fall short when they are deployed in real systems. One main reason stems from the difficulty of fully understanding the users' natural language questions. In this paper, we include human in the loop and present a novel parser-independent interactive approach (PIIA) that interacts with users using multi-choice questions and can easily work with arbitrary parsers. Experiments were conducted on two cross-domain datasets, the WikiSQL and the more complex Spider, with five state-of-the-art parsers. These demonstrated that PIIA is capable of enhancing the text-to-SQL performance with limited interaction turns by using both simulation and human evaluation.