
TL;DR
This paper advocates for a cultural and technical shift in computer science towards making experiments fully replicable through the use of virtual machines, emphasizing the importance of standard practices and tools for replication.
Contribution
It introduces the idea that providing virtual machine images is the key to achieving true experiment replication in computer science.
Findings
Virtual machines ensure experiment reproducibility.
Current practices fall short of true replication.
Tools and repositories are essential for widespread adoption.
Abstract
Replication of scientific experiments is critical to the advance of science. Unfortunately, the discipline of Computer Science has never treated replication seriously, even though computers are very good at doing the same thing over and over again. Not only are experiments rarely replicated, they are rarely even replicable in a meaningful way. Scientists are being encouraged to make their source code available, but this is only a small step. Even in the happy event that source code can be built and run successfully, running code is a long way away from being able to replicate the experiment that code was used for. I propose that the discipline of Computer Science must embrace replication of experiments as standard practice. I propose that the only credible technique to make experiments truly replicable is to provide copies of virtual machines in which the experiments are validated to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
