Darknet and Deepnet Mining for Proactive Cybersecurity Threat Intelligence

TL;DR

This paper introduces an operational system that mines darknet and deepnet platforms to gather cyber threat intelligence, utilizing data mining and machine learning to identify emerging threats and provide valuable alerts to cyber-defenders.

Contribution

The paper presents a novel system for collecting and analyzing cyber threat information from darknet and deepnet sites, enhancing threat detection with machine learning techniques.

Findings

Average of 305 threat warnings collected weekly

Recall of 92% for marketplace products

Recall of 80% for malicious forum discussions

Abstract

In this paper, we present an operational system for cyber threat intelligence gathering from various social platforms on the Internet particularly sites on the darknet and deepnet. We focus our attention to collecting information from hacker forum discussions and marketplaces offering products and services focusing on malicious hacking. We have developed an operational system for obtaining information from these sites for the purposes of identifying emerging cyber threats. Currently, this system collects on average 305 high-quality cyber threat warnings each week. These threat warnings include information on newly developed malware and exploits that have not yet been deployed in a cyber-attack. This provides a significant service to cyber-defenders. The system is significantly augmented through the use of various data mining and machine learning techniques. With the use of machine learning models, we are able to recall 92% of products in marketplaces and 80% of discussions on forums relating to malicious hacking with high precision. We perform preliminary analysis on the data collected, demonstrating its application to aid a security expert for better threat analysis.