PASTRAMI: Performance Assessment of SofTware Routers Addressing Measurement Instability

TL;DR

PASTRAMI introduces a methodology to evaluate software router stability in virtualized environments, ensuring accurate performance measurement by identifying and selecting stable configurations.

Contribution

It provides a systematic approach to distinguish stable from unstable environments, improving the reliability of network performance assessments in virtualized setups.

Findings

Different Linux versions show varied behaviors affecting measurement accuracy.

Some Linux versions exhibit packet loss at low loads, compromising evaluation.

The methodology helps select configurations with minimal measurement variability.

Abstract

Virtualized environments offer a flexible and scalable platform for evaluating network performance, but they can introduce significant variability that complicates accurate measurement. This paper presents PASTRAMI, a methodology designed to assess the stability of software routers, which is critical to accurately evaluate performance metrics such as the Partial Drop Rate at 0.5% (PDR@0.5%). While PDR@0.5% is a key metric to assess packet processing capabilities of a software router, its reliable evaluation depends on consistent router performance with minimal measurement variability. Our research reveals that different Linux versions exhibit distinct behaviors, with some demonstrating non-negligible packet loss even at low loads and high variability in loss measurements, rendering them unsuitable for accurate performance assessments. This paper proposes a systematic approach to differentiate between stable and unstable environments, offering practical guidance on selecting suitable configurations for robust networking performance evaluations in virtualized environments.