Web Scale Graph Mining for Cyber Threat Intelligence

TL;DR

TITAN is a scalable, real-time graph mining framework integrated into Microsoft USOP that enhances cyber threat intelligence, detection, and disruption capabilities across large-scale networks with high accuracy and efficiency.

Contribution

The paper introduces TITAN, a novel industry-scale graph mining system that dynamically models, updates, and propagates threat intelligence at unprecedented speed and scale.

Findings

Achieved an average macro-F1 score of 0.89 in threat detection.

Enabled a 6x increase in non-file threat intelligence.

Increased incident disruption rate by 21%, reduced disruption time by 1.9x.

Abstract

Defending against today's increasingly sophisticated and large-scale cyberattacks demands accurate, real-time threat intelligence. Traditional approaches struggle to scale, integrate diverse telemetry, and adapt to a constantly evolving security landscape. We introduce Threat Intelligence Tracking via Adaptive Networks (TITAN), an industry-scale graph mining framework that generates cyber threat intelligence at unprecedented speed and scale. TITAN introduces a suite of innovations specifically designed to address the complexities of the modern security landscape, including: (1) a dynamic threat intelligence graph that maps the intricate relationships between millions of entities, incidents, and organizations; (2) real-time update mechanisms that automatically decay and prune outdated intel; (3) integration of security domain knowledge to bootstrap initial reputation scores; and (4) reputation propagation algorithms that uncover hidden threat actor infrastructure. Integrated into Microsoft Unified Security Operations Platform (USOP), which is deployed across hundreds of thousands of organizations worldwide, TITAN's threat intelligence powers key detection and disruption capabilities. With an impressive average macro-F1 score of 0.89 and a precision-recall AUC of 0.94, TITAN identifies millions of high-risk entities each week, enabling a 6x increase in non-file threat intelligence. Since its deployment, TITAN has increased the product's incident disruption rate by a remarkable 21%, while reducing the time to disrupt by a factor of 1.9x, and maintaining 99% precision, as confirmed by customer feedback and thorough manual evaluation by security experts--ultimately saving customers from costly security breaches.