Exploiting Latent Attack Semantics for Intelligent Malware Detection

TL;DR

This paper introduces Shape GD, a novel malware detection system that leverages the structure and statistical shape of neighborhood-level features to improve early and robust detection of malware in noisy environments.

Contribution

Shape GD uniquely combines structural and statistical insights to aggregate weak local detectors into a robust global anomaly detector, enhancing early malware detection.

Findings

Detects malware early with high accuracy (~100 infected nodes in 100K system)

Achieves near-perfect true positive rate (~100%) and low false positive rate (~1%)

Outperforms prior methods by analyzing feature vector shapes rather than alert streams

Abstract

Behavioral malware detectors promise to expose previously unknown malware and are an important security primitive. However, even the best behavioral detectors suffer from high false positives and negatives. In this paper, we address the challenge of aggregating weak per-device behavioral detectors in noisy communities (i.e., ones that produce alerts at unpredictable rates) into an accurate and robust global anomaly detector (GD). Our system - Shape GD - combines two insights: Structural: actions such as visiting a website (waterhole attack) or membership in a shared email thread (phishing attack) by nodes correlate well with malware spread, and create dynamic neighborhoods of nodes that were exposed to the same attack vector; and Statistical: feature vectors corresponding to true and false positives of local detectors have markedly different conditional distributions. We use neighborhoods to amplify the transient low-dimensional structure that is latent in high-dimensional feature vectors - but neighborhoods vary unpredictably, and we use shape to extract robust neighborhood-level features that identify infected neighborhoods. Unlike prior works that aggregate local detectors' alert bitstreams or cluster the feature vectors, Shape GD analyzes the feature vectors that led to local alerts (alert-FVs) to separate true and false positives. Shape GD first filters these alert-FVs into neighborhoods and efficiently maps a neighborhood's alert-FVs' statistical shapes into a scalar score. Shape GD then acts as a neighborhood level anomaly detector - training on benign program traces to learn the ShapeScore of false positive neighborhoods, and classifying neighborhoods with anomalous ShapeScores as malicious. Shape GD detects malware early (~100 infected nodes in a ~100K node system for waterhole and ~10 of 1000 for phishing) and robustly (with ~100% global TP and ~1% global FP rates).