Integrating Regular Expressions with Neural Networks via DFA

TL;DR

This paper presents a hybrid approach that integrates human-designed regular expression rules into neural networks using minimal deterministic finite automatons, improving performance on intent classification especially with limited training data.

Contribution

The paper introduces a novel method to incorporate regular expression-based rules into neural networks via MDFAs, enhancing model performance with small datasets.

Findings

Achieves superior accuracy on ATIS intent classification with limited data

Demonstrates the effectiveness of MDFA as an intermediate rule representation

Outperforms existing RE-neural network combination methods

Abstract

Human-designed rules are widely used to build industry applications. However, it is infeasible to maintain thousands of such hand-crafted rules. So it is very important to integrate the rule knowledge into neural networks to build a hybrid model that achieves better performance. Specifically, the human-designed rules are formulated as Regular Expressions (REs), from which the equivalent Minimal Deterministic Finite Automatons (MDFAs) are constructed. We propose to use the MDFA as an intermediate model to capture the matched RE patterns as rule-based features for each input sentence and introduce these additional features into neural networks. We evaluate the proposed method on the ATIS intent classification task. The experiment results show that the proposed method achieves the best performance compared to neural networks and four other methods that combine REs and neural networks when the training dataset is relatively small.