BOTracle: A framework for Discriminating Bots and Humans

TL;DR

BOTracle introduces a comprehensive framework for detecting bots in high-traffic environments by comparing heuristic, technical, and behavioral methods, demonstrating superior accuracy over existing approaches using real-world e-commerce data.

Contribution

This work presents a novel evaluation of three distinct bot detection methods on large-scale real-world data, highlighting the effectiveness of behavior-based detection.

Findings

Behavioral method achieves over 98% precision and recall.

Behavioral detection outperforms Botcha in accuracy metrics.

Heuristic and technical methods provide rapid detection options.

Abstract

Bots constitute a significant portion of Internet traffic and are a source of various issues across multiple domains. Modern bots often become indistinguishable from real users, as they employ similar methods to browse the web, including using real browsers. We address the challenge of bot detection in high-traffic scenarios by analyzing three distinct detection methods. The first method operates on heuristics, allowing for rapid detection. The second method utilizes, well known, technical features, such as IP address, window size, and user agent. It serves primarily for comparison with the third method. In the third method, we rely solely on browsing behavior, omitting all static features and focusing exclusively on how clients behave on a website. In contrast to related work, we evaluate our approaches using real-world e-commerce traffic data, comprising 40 million monthly page visits. We further compare our methods against another bot detection approach, Botcha, on the same dataset. Our performance metrics, including precision, recall, and AUC, reach 98 percent or higher, surpassing Botcha.