SCADE: Scalable Framework for Anomaly Detection in High-Performance System

TL;DR

SCADE is a scalable, unsupervised framework that combines statistical models and local analysis to detect command-line anomalies in high-performance computing environments with high accuracy and low false positives.

Contribution

The paper introduces SCADE, a novel anomaly detection framework that integrates global statistical models with local context analysis for improved cybersecurity in HPC systems.

Findings

Achieves above 98% SNR in anomaly detection.

Effectively minimizes false positives in low SNR environments.

Demonstrates scalability and robustness in enterprise settings.

Abstract

As command-line interfaces remain integral to high-performance computing environments, the risk of exploitation through stealthy and complex command-line abuse grows. Conventional security solutions struggle to detect these anomalies due to their context-specific nature, lack of labeled data, and the prevalence of sophisticated attacks like Living-off-the-Land (LOL). To address this gap, we introduce the Scalable Command-Line Anomaly Detection Engine (SCADE), a framework that combines global statistical models with local context-specific analysis for unsupervised anomaly detection. SCADE leverages novel statistical methods, including BM25 and Log Entropy, alongside dynamic thresholding to adaptively detect rare, malicious command-line patterns in low signal-to-noise ratio (SNR) environments. Experimental results show that SCADE achieves above 98% SNR in identifying anomalous behavior while minimizing false positives. Designed for scalability and precision, SCADE provides an innovative, metadata-enriched approach to anomaly detection, offering a robust solution for cybersecurity in high-computation environments. This work presents SCADE's architecture, detection methodology, and its potential for enhancing anomaly detection in enterprise systems. We argue that SCADE represents a significant advancement in unsupervised anomaly detection, offering a robust, adaptive framework for security analysts and researchers seeking to enhance detection accuracy in high-computation environments.