Learning from Uncurated Regular Expressions

TL;DR

This paper proposes a novel method for learning from uncurated, publicly available regular expressions to improve feature extraction and classification tasks, offering a scalable alternative to manual or data-driven approaches.

Contribution

It introduces a new approach that leverages uncurated regexes from public repositories for feature extraction, reducing overhead and broadening applicability.

Findings

Model trained on uncurated regexes performs competitively on semantic classification.

Feature extraction code is significantly smaller than existing methods.

Approach enables unsupervised learning using uncurated regex data.

Abstract

Significant work has been done on learning regular expressions from a set of data values. Depending on the domain, this approach can be very successful. However, significant time is required to learn these expressions and the resulting expressions can become either very complex or inaccurate in the presence of dirty data. The alternative of manually writing regular expressions becomes unattractive when faced with a large number of values that must be matched. As an alternative, we propose learning from a large corpus of manually authored, but uncurated regular expressions mined from a public repository. The advantage of this approach is that we are able to extract salient features from a set of strings with limited overhead to feature engineering. Since the set of regular expressions covers a wide range of application domains, we expect them to be widely applicable. To demonstrate the potential effectiveness of our approach, we train a model using the extracted corpus of regular expressions for the class of semantic type classification. While our approach yields results that are overall inferior to the state-of-the-art, our feature extraction code is an order of magnitude smaller, and our model outperforms a popular existing approach on some classes. We also demonstrate the possibility of using uncurated regular expressions for unsupervised learning.