Database Entity Recognition with Data Augmentation and Deep Learning

TL;DR

This paper introduces a new benchmark, data augmentation method, and T5-based model for improving database entity recognition in natural language queries, demonstrating significant performance gains over existing methods.

Contribution

The paper presents a novel data augmentation technique, a specialized T5-based entity recognition model, and a human-annotated benchmark for DB-ER in NLQ, advancing the state-of-the-art in this task.

Findings

Data augmentation improves precision and recall by over 10%.

Fine-tuning T5 backbone boosts metrics by 5-10%.

Our model outperforms two state-of-the-art NER taggers.

Abstract

This paper addresses the challenge of Database Entity Recognition (DB-ER) in Natural Language Queries (NLQ). We present several key contributions to advance this field: (1) a human-annotated benchmark for DB-ER task, derived from popular text-to-sql benchmarks, (2) a novel data augmentation procedure that leverages automatic annotation of NLQs based on the corresponding SQL queries which are available in popular text-to-SQL benchmarks, (3) a specialized language model based entity recognition model using T5 as a backbone and two down-stream DB-ER tasks: sequence tagging and token classification for fine-tuning of backend and performing DB-ER respectively. We compared our DB-ER tagger with two state-of-the-art NER taggers, and observed better performance in both precision and recall for our model. The ablation evaluation shows that data augmentation boosts precision and recall by over 10%, while fine-tuning of the T5 backbone boosts these metrics by 5-10%.