Cybersecurity Software Tool Evaluation Using a 'Perfect' Network Model

TL;DR

This paper introduces a 'perfect' network model as a standardized benchmark for evaluating cybersecurity tools, enabling consistent testing and analysis under controlled conditions.

Contribution

It proposes a novel methodology using a 'perfect' network to standardize and improve cybersecurity tool evaluation processes.

Findings

Provides a consistent benchmark for testing cybersecurity tools.

Allows controlled testing with varying levels of error and noise.

Facilitates comparison across different cybersecurity tools.

Abstract

Cybersecurity software tool evaluation is difficult due to the inherently adversarial nature of the field. A penetration testing (or offensive) tool must be tested against a viable defensive adversary and a defensive tool must, similarly, be tested against a viable offensive adversary. Characterizing the tool's performance inherently depends on the quality of the adversary, which can vary from test to test. This paper proposes the use of a 'perfect' network, representing computing systems, a network and the attack pathways through it as a methodology to use for testing cybersecurity decision-making tools. This facilitates testing by providing a known and consistent standard for comparison. It also allows testing to include researcher-selected levels of error, noise and uncertainty to evaluate cybersecurity tools under these experimental conditions.