Data Agnostic RoBERTa-based Natural Language to SQL Query Generation

TL;DR

This paper presents a data privacy-focused NL2SQL model using RoBERTa embeddings and data-agnostic knowledge vectors, achieving 76.7% accuracy without relying on table data during training.

Contribution

It introduces a novel approach that eliminates the need for table data during training, enabling zero-shot learning for NL2SQL tasks.

Findings

Achieved 76.7% test set execution accuracy.

Eliminated dependency on table data during training.

Enabled zero-shot learning based on schema and question.

Abstract

Relational databases are among the most widely used architectures to store massive amounts of data in the modern world. However, there is a barrier between these databases and the average user. The user often lacks the knowledge of a query language such as SQL required to interact with the database. The NL2SQL task aims at finding deep learning approaches to solve this problem by converting natural language questions into valid SQL queries. Given the sensitive nature of some databases and the growing need for data privacy, we have presented an approach with data privacy at its core. We have passed RoBERTa embeddings and data-agnostic knowledge vectors into LSTM based submodels to predict the final query. Although we have not achieved state of the art results, we have eliminated the need for the table data, right from the training of the model, and have achieved a test set execution accuracy of 76.7%. By eliminating the table data dependency while training we have created a model capable of zero shot learning based on the natural language question and table schema alone.