Reproducible and Adaptable Log Data Generation for Sound Cybersecurity Experiments

TL;DR

This paper introduces SOCBED, a reproducible and adaptable testbed for generating realistic cybersecurity log data, addressing reproducibility and customization challenges in cybersecurity research.

Contribution

The paper presents SOCBED, the first testbed designed specifically for reproducible and adaptable cybersecurity log data generation, facilitating valid and controlled experiments.

Findings

SOCBED enables reproduction of testbed instances on commodity hardware.

The generated log data supports realistic cybersecurity experiments.

Experiments using SOCBED are validated as valid, controlled, and reproducible.

Abstract

Artifacts such as log data and network traffic are fundamental for cybersecurity research, e.g., in the area of intrusion detection. Yet, most research is based on artifacts that are not available to others or cannot be adapted to own purposes, thus making it difficult to reproduce and build on existing work. In this paper, we identify the challenges of artifact generation with the goal of conducting sound experiments that are valid, controlled, and reproducible. We argue that testbeds for artifact generation have to be designed specifically with reproducibility and adaptability in mind. To achieve this goal, we present SOCBED, our proof-of-concept implementation and the first testbed with a focus on generating realistic log data for cybersecurity experiments in a reproducible and adaptable manner. SOCBED enables researchers to reproduce testbed instances on commodity computers, adapt them according to own requirements, and verify their correct functionality. We evaluate SOCBED with an exemplary, practical experiment on detecting a multi-step intrusion of an enterprise network and show that the resulting experiment is indeed valid, controlled, and reproducible. Both SOCBED and the log dataset underlying our evaluation are freely available.