Machine Learning for Machine Data from a CATI Network

TL;DR

This paper demonstrates high-accuracy machine learning methods, combining NLP and data-mining, to predict rare error events in large-scale machine log data from a CATI network, without source code access.

Contribution

It introduces a simple feature preallocation approach with naive Bayes classifiers for imbalanced data, applicable to various critical event predictions.

Findings

Effective prediction of rare errors in large log data

NLP and data-mining techniques improve classification accuracy

Method generalizes to cyberattack detection and other domains

Abstract

This is a machine learning application paper involving big data. We present high-accuracy prediction methods of rare events in semi-structured machine log files, which are produced at high velocity and high volume by NORC's computer-assisted telephone interviewing (CATI) network for conducting surveys. We judiciously apply natural language processing (NLP) techniques and data-mining strategies to train effective learning and prediction models for classifying uncommon error messages in the log---without access to source code, updated documentation or dictionaries. In particular, our simple but effective approach of features preallocation for learning from imbalanced data coupled with naive Bayes classifiers can be conceivably generalized to supervised or semi-supervised learning and prediction methods for other critical events such as cyberattack detection.