Efficient classification using parallel and scalable compressed model and Its application on intrusion detection

TL;DR

This paper introduces a scalable, compressed classification model for intrusion detection that combines attribute and data reduction techniques with parallel processing, significantly speeding up detection with minimal accuracy loss.

Contribution

It presents a novel scalable compression method integrating horizontal and vertical compression with MapReduce parallelization for intrusion detection.

Findings

Speeded up detection by up to 184 times

Achieved less than 1% accuracy loss

Validated on KDD99 and CMDC2012 datasets

Abstract

In order to achieve high efficiency of classification in intrusion detection, a compressed model is proposed in this paper which combines horizontal compression with vertical compression. OneR is utilized as horizontal com-pression for attribute reduction, and affinity propagation is employed as vertical compression to select small representative exemplars from large training data. As to be able to computationally compress the larger volume of training data with scalability, MapReduce based parallelization approach is then implemented and evaluated for each step of the model compression process abovementioned, on which common but efficient classification methods can be directly used. Experimental application study on two publicly available datasets of intrusion detection, KDD99 and CMDC2012, demonstrates that the classification using the compressed model proposed can effectively speed up the detection procedure at up to 184 times, most importantly at the cost of a minimal accuracy difference with less than 1% on average.