Assessing virtualization effects in simulations of distributed robotics

TL;DR

This paper investigates how different virtualization strategies and operating systems impact the accuracy of distributed robotics simulations, highlighting significant effects that influence research reproducibility.

Contribution

It provides empirical evidence that virtualization choices significantly affect simulation performance, emphasizing the need for careful selection to ensure repeatability.

Findings

Virtualization strategy significantly affects simulation accuracy.

Operating systems can amplify virtualization effects.

Results support cautious use of virtualization for reproducible robotics research.

Abstract

In this work, our aim is to identify whether the choice of virtualization strategy influences the performance of simulations in robotics. Performance is quantified in the error between a reference trajectory and the actual trajectory for the ball moving along the surface of a smooth plate. The two-sample Kolmogorov-Smirnov test is used to assess significance of variations in performance under the different experimental settings. Our results show that the selection of virtualization technology does have a significant effect on simulation, and moreover this effect can be amplified by the use of some operating systems. While these results are a strong cause for caution, they also provide reason for optimism for those considering 'repeatable robotics research' using virtualization.