SCADS: A Scalable Approach Using Spark in Cloud for Host-based Intrusion Detection System with System Calls

TL;DR

This paper introduces SCADS, a scalable cloud-based host intrusion detection system leveraging Apache Spark to handle large-scale system call data efficiently, improving detection performance in data center environments.

Contribution

The paper presents a novel scalable approach using Spark in the cloud for host-based intrusion detection with system calls, addressing scalability limitations of traditional methods.

Findings

Enhanced detection efficiency demonstrated in experiments

Scalable Spark algorithms effectively handle large system call traces

Applicable to next-generation intrusion detection systems

Abstract

Following the current big data trend, the scale of real-time system call traces generated by Linux applications in a contemporary data center may increase excessively. Due to the deficiency of scalability, it is challenging for traditional host-based intrusion detection systems deployed on every single host to collect, maintain, and manipulate those large-scale accumulated system call traces. It is inflexible to build data mining models on one physical host that has static computing capability and limited storage capacity. To address this issue, we propose SCADS, a corresponding solution using Apache Spark in the Google cloud environment. A set of Spark algorithms are developed to achieve the computational scalability. The experiment results demonstrate that the efficiency of intrusion detection can be enhanced, which indicates that the proposed method can apply to the design of next-generation host-based intrusion detection systems with system calls.