REPETITA: Repeatable Experiments for Performance Evaluation of Traffic-Engineering Algorithms

TL;DR

REPETITA is a software framework designed to improve the reproducibility of traffic engineering algorithm experiments by providing datasets, implementation of algorithms, and tools for comparison and analysis.

Contribution

It introduces a comprehensive, extendable framework with datasets and algorithms to facilitate reproducible and comparable traffic engineering experiments.

Findings

Successfully reproduces literature results

Facilitates new analyses of TE algorithms

Openly available for community use and extension

Abstract

In this paper, we propose a pragmatic approach to improve reproducibility of experimental analyses of traffic engineering (TE) algorithms, whose implementation, evaluation and comparison are currently hard to replicate. Our envisioned goal is to enable universally-checkable experiments of existing and future TE algorithms. We describe the design and implementation of REPETITA, a software framework that implements common TE functions, automates experimental setup, and eases comparisons (in terms of solution quality, execution time, etc.) of TE algorithms. In its current version, REPETITA includes (i) a dataset for repeatable experiments, consisting of more than 250 real network topologies with complete bandwidth and delay information as well as associated traffic matrices; and (ii) the implementation of state-of-the-art algorithms for intra-domain TE with IGP weight tweaking and Segment Routing optimization. We showcase how our framework can successfully reproduce results described in the literature, and ease new analyses of qualitatively-diverse TE algorithms. We publicly release our REPETITA implementation, hoping that the community will consider it as a demonstration of feasibility, an incentive and an initial code basis for improving experiment reproducibility: Its plugin-oriented architecture indeed makes REPETITA easy to extend with new datasets, algorithms, TE primitives and analyses. We therefore invite the research community to use and contribute to our released code and dataset.