PHASE: Passive Human Activity Simulation Evaluation

TL;DR

PHASE is a machine learning framework that passively analyzes network logs to distinguish human from non-human activity with high accuracy, improving the realism of synthetic user personas in cybersecurity simulations.

Contribution

The paper introduces PHASE, a novel passive analysis method using Zeek logs and DNS-based labeling to evaluate and enhance behavioral fidelity of synthetic users.

Findings

PHASE achieves over 90% accuracy in classifying human activity.

The approach uncovers behavioral signatures that differentiate genuine users from bots.

Revised configurations improve the realism of synthetic user personas.

Abstract

Cybersecurity simulation environments, such as cyber ranges, honeypots, and sandboxes, require realistic human behavior to be effective, yet no quantitative method exists to assess the behavioral fidelity of synthetic user personas. This paper presents PHASE (Passive Human Activity Simulation Evaluation), a machine learning framework that analyzes Zeek connection logs and distinguishes human from non-human activity with over 90\% accuracy. PHASE operates entirely passively, relying on standard network monitoring without any user-side instrumentation or visible signs of surveillance. All network activity used for machine learning is collected via a Zeek network appliance to avoid introducing unnecessary network traffic or artifacts that could disrupt the fidelity of the simulation environment. The paper also proposes a novel labeling approach that utilizes local DNS records to classify network traffic, thereby enabling machine learning analysis. Furthermore, we apply SHAP (SHapley Additive exPlanations) analysis to uncover temporal and behavioral signatures indicative of genuine human users. In a case study, we evaluate a synthetic user persona and identify distinct non-human patterns that undermine behavioral realism. Based on these insights, we develop a revised behavioral configuration that significantly improves the human-likeness of synthetic activity yielding a more realistic and effective synthetic user persona.