Automated, Unsupervised, and Auto-parameterized Inference of Data Patterns and Anomaly Detection

TL;DR

This paper presents RIOLU, an automated, unsupervised method for inferring data patterns and detecting anomalies that outperforms existing approaches in accuracy and efficiency without requiring labeled data or manual configuration.

Contribution

RIOLU introduces a fully automated, parameter-free approach for pattern inference and anomaly detection in unlabeled data, surpassing state-of-the-art performance.

Findings

Achieves a 97.2% F1 score in pattern inference.

Up to 800.4% improvement in anomaly detection F1 score.

Outperforms ChatGPT in accuracy and inference time.

Abstract

With the advent of data-centric and machine learning (ML) systems, data quality is playing an increasingly critical role in ensuring the overall quality of software systems. Data preparation, an essential step towards high data quality, is known to be a highly effort-intensive process. Although prior studies have dealt with one of the most impacting issues, data pattern violations, these studies usually require data-specific configurations (i.e., parameterized) or use carefully curated data as learning examples (i.e., supervised), relying on domain knowledge and deep understanding of the data, or demanding significant manual effort. In this paper, we introduce RIOLU: Regex Inferencer auto-parameterized Learning with Uncleaned data. RIOLU is fully automated, automatically parameterized, and does not need labeled samples. RIOLU can generate precise patterns from datasets in various domains, with a high F1 score of 97.2%, exceeding the state-of-the-art baseline. In addition, according to our experiment on five datasets with anomalies, RIOLU can automatically estimate a data column's error rate, draw normal patterns, and predict anomalies from unlabeled data with higher performance (up to 800.4% improvement in terms of F1) than the state-of-the-art baseline, even outperforming ChatGPT in terms of both accuracy (12.3% higher F1) and efficiency (10% less inference time). A variant of RIOLU, with user guidance, can further boost its precision, with up to 37.4% improvement in terms of F1. Our evaluation in an industrial setting further demonstrates the practical benefits of RIOLU.