Marrying up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding

TL;DR

This paper explores combining regular expressions with neural networks to improve spoken language understanding tasks, especially when training data is limited, demonstrating significant performance gains.

Contribution

It introduces novel methods to integrate regular expressions into neural networks, enhancing NLP performance with small datasets.

Findings

Significant improvement over neural networks without REs.

Effective in low-data scenarios for intent detection and slot filling.

Demonstrates the value of hybrid RE-NN models in NLP.

Abstract

The success of many natural language processing (NLP) tasks is bound by the number and quality of annotated data, but there is often a shortage of such training data. In this paper, we ask the question: "Can we combine a neural network (NN) with regular expressions (RE) to improve supervised learning for NLP?". In answer, we develop novel methods to exploit the rich expressiveness of REs at different levels within a NN, showing that the combination significantly enhances the learning effectiveness when a small number of training examples are available. We evaluate our approach by applying it to spoken language understanding for intent detection and slot filling. Experimental results show that our approach is highly effective in exploiting the available training data, giving a clear boost to the RE-unaware NN.