PACT: Reducing Alert Fatigue in Low-Prevalence SOC Streams with Triggered Active Learning

TL;DR

PACT is a novel adaptive controller that reduces false positives and analyst workload in low-prevalence security streams by combining triggered active learning with hybrid sampling strategies.

Contribution

It introduces PACT, a Pareto-aware triggered active learning method that significantly decreases false positives and query costs in SOC alert management.

Findings

PACT reduces false-positive burden by 43% and 21% on two benchmarks.

It uses 3.8x and 5.2x fewer analyst queries than uniform-random updating.

Ablation shows acquisition improves beyond trigger timing alone.

Abstract

Security operations centers face persistent alert fatigue: in low-prevalence streams, even low false-positive rates generate substantial investigation load, while aggregate F1 scores obscure analyst burden. We introduce PACT, a Pareto-aware controller for triggered active learning, which wraps an already-deployed frozen XGBoost-Focal screener with an adaptive windowing score-shift trigger and a hybrid acquisition rule combining threshold-relative uncertainty with high-score sampling. On two public low-prevalence benchmarks, AIT-ADS (AIT Alert Data Set), and BOTSv1 (Boss of the SOC version 1), PACT attains the lowest benign-normalized false-positive (FP) burden among the adaptive methods tested. It reduces burden by 43% and 21%, respectively, relative to a frozen baseline, while using 3.8x and 5.2x fewer analyst queries than periodic uniform-random updating. A matched-trigger ablation controls trigger timing and shows that acquisition contributes beyond timing alone, at the cost of approximately ten percentage points of positive-window recall under free-running triggers. A frozen threshold-only baseline pushes FP lower still but collapses BOTSv1 recall by 55 percentage points. Under the evaluated workload assumptions, pure FP minimization trades unacceptable recall for that lower burden.