Carbon Filter: Real-time Alert Triage Using Large Scale Clustering and Fast Search
Jonathan Oliver, Raghav Batta, Adam Bates, Muhammad Adil Inam, Shelly, Mehta, Shugao Xia

TL;DR
Carbon Filter is a scalable, real-time alert triage system that uses statistical learning and fast search algorithms to significantly reduce false alerts in security operations, improving efficiency without losing detection quality.
Contribution
The paper introduces Carbon Filter, a novel system that leverages process context and fast search techniques to efficiently triage millions of security alerts in real-time.
Findings
Achieved a 6-fold increase in Signal-to-Noise ratio.
Scales to process up to 20 million alerts per hour.
Reduces manual review workload significantly.
Abstract
"Alert fatigue" is one of the biggest challenges faced by the Security Operations Center (SOC) today, with analysts spending more than half of their time reviewing false alerts. Endpoint detection products raise alerts by pattern matching on event telemetry against behavioral rules that describe potentially malicious behavior, but can suffer from high false positives that distract from actual attacks. While alert triage techniques based on data provenance may show promise, these techniques can take over a minute to inspect a single alert, while EDR customers may face tens of millions of alerts per day; the current reality is that these approaches aren't nearly scalable enough for production environments. We present Carbon Filter, a statistical learning based system that dramatically reduces the number of alerts analysts need to manually review. Our approach is based on the observation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAnomaly Detection Techniques and Applications · Fire Detection and Safety Systems · Air Quality Monitoring and Forecasting
