Don't Repeat Yourself: Seamless Execution and Analysis of Extensive Network Experiments

TL;DR

This paper introduces MACI, a comprehensive framework that streamlines the management, execution, and analysis of large-scale network experiments, enhancing efficiency and reproducibility across various communication research projects.

Contribution

MACI is the first bespoke framework that integrates simulators and analysis tools for scalable, reproducible, and systematic network experiments, addressing repetitive scripting issues.

Findings

MACI significantly improved experiment efficiency and reproducibility.

MACI was essential in diverse research projects including DASH streaming and TCP scheduler development.

MACI is publicly available for the research community.

Abstract

This paper presents MACI, the first bespoke framework for the management, the scalable execution, and the interactive analysis of a large number of network experiments. Driven by the desire to avoid repetitive implementation of just a few scripts for the execution and analysis of experiments, MACI emerged as a generic framework for network experiments that significantly increases efficiency and ensures reproducibility. To this end, MACI incorporates and integrates established simulators and analysis tools to foster rapid but systematic network experiments. We found MACI indispensable in all phases of the research and development process of various communication systems, such as i) an extensive DASH video streaming study, ii) the systematic development and improvement of Multipath TCP schedulers, and iii) research on a distributed topology graph pattern matching algorithm. With this work, we make MACI publicly available to the research community to advance efficient and reproducible network experiments.