WebEye - Automated Collection of Malicious HTTP Traffic

TL;DR

WebEye is an automated framework that generates realistic web traffic data, classifies it as malicious or benign, and provides datasets for training and evaluating machine learning-based malware detection systems.

Contribution

WebEye introduces an autonomous system for collecting, enriching, and classifying web traffic data, facilitating large-scale malware detection research and development.

Findings

WebEye successfully collects and classifies large datasets of malicious and benign web traffic.

Datasets generated by WebEye improve training of machine learning malware detection algorithms.

WebEye provides a valuable tool for benchmarking and enhancing anti-malware solutions.

Abstract

With malware detection techniques increasingly adopting machine learning approaches, the creation of precise training sets becomes more and more important. Large data sets of realistic web traffic, correctly classified as benign or malicious are needed, not only to train classic and deep learning algorithms, but also to serve as evaluation benchmarks for existing malware detection products. Interestingly, despite the vast number and versatility of threats a user may encounter when browsing the web, actual malicious content is often hard to come by, since prerequisites such as browser and operating system type and version must be met in order to receive the payload from a malware distributing server. In combination with privacy constraints on data sets of actual user traffic, it is difficult for researchers and product developers to evaluate anti-malware solutions against large-scale data sets of realistic web traffic. In this paper we present WebEye, a framework that autonomously creates realistic HTTP traffic, enriches recorded traffic with additional information, and classifies records as malicious or benign, using different classifiers. We are using WebEye to collect malicious HTML and JavaScript and show how datasets created with WebEye can be used to train machine learning based malware detection algorithms. We regard WebEye and the data sets it creates as a tool for researchers and product developers to evaluate and improve their AI-based anti-malware solutions against large-scale benchmarks.