Efficient Deployment of Conversational Natural Language Interfaces over Databases

TL;DR

This paper introduces a novel system that accelerates the collection of training data for natural language-to-query models by generating conversational multi-turn datasets, improving training efficiency for chatbot-based question-answering over databases.

Contribution

The work presents a new method for generating conversational multi-turn datasets to facilitate training of NL-to-query models, addressing data scarcity issues.

Findings

Successfully trained models on SQL and SPARQL datasets

Demonstrated improved data collection efficiency

Showcased adaptability across query languages

Abstract

Many users communicate with chatbots and AI assistants in order to help them with various tasks. A key component of the assistant is the ability to understand and answer a user's natural language questions for question-answering (QA). Because data can be usually stored in a structured manner, an essential step involves turning a natural language question into its corresponding query language. However, in order to train most natural language-to-query-language state-of-the-art models, a large amount of training data is needed first. In most domains, this data is not available and collecting such datasets for various domains can be tedious and time-consuming. In this work, we propose a novel method for accelerating the training dataset collection for developing the natural language-to-query-language machine learning models. Our system allows one to generate conversational multi-term data, where multiple turns define a dialogue session, enabling one to better utilize chatbot interfaces. We train two current state-of-the-art NL-to-QL models, on both an SQL and SPARQL-based datasets in order to showcase the adaptability and efficacy of our created data.