CARONTE: Crawling Adversarial Resources Over Non-Trusted, High-Profile Environments

TL;DR

CARONTE is a semi-automatic tool designed to covertly crawl underground criminal forums, effectively learning forum structures and extracting data while minimizing detection risk and avoiding large dataset requirements.

Contribution

It introduces a novel semi-automatic approach for parsing and data extraction from high-profile underground forums with low detection risk.

Findings

Successfully applied to four underground forums

Generated network traffic comparable to human users

Outperformed state-of-the-art web-crawling tools in stealth

Abstract

The monitoring of underground criminal activities is often automated to maximize the data collection and to train ML models to automatically adapt data collection tools to different communities. On the other hand, sophisticated adversaries may adopt crawling-detection capabilities that may significantly jeopardize researchers' opportunities to perform the data collection, for example by putting their accounts under the spotlight and being expelled from the community. This is particularly undesirable in prominent and high-profile criminal communities where entry costs are significant (either monetarily or for example for background checking or other trust-building mechanisms). This paper presents CARONTE, a tool to semi-automatically learn virtually any forum structure for parsing and data-extraction, while maintaining a low profile for the data collection and avoiding the requirement of collecting massive datasets to maintain tool scalability. We showcase the tool against four underground forums, and compare the network traffic it generates (as seen from the adversary's position, i.e. the underground community's server) against state-of-the-art tools for web-crawling as well as human users.