Publishing Identifiable Experiment Code And Configuration Is Important, Good and Easy

TL;DR

This paper emphasizes the importance of publishing complete experiment code, configurations, and data scripts in robotics to enhance reproducibility, proposing the use of unique identifiers and discussing current practices and incentives.

Contribution

It advocates for publishing identifiable code packages with unique IDs and reviews best practices and reward structures for reproducibility in robotics research.

Findings

Publishing code improves reproducibility in robotics.

Unique identifiers help link code to published results.

Current reward systems for code sharing are insufficient.

Abstract

We argue for the value of publishing the exact code, configuration and data processing scripts used to produce empirical work in robotics. In particular, we recommend publishing a unique identifier for the code package in the paper itself, as a promise to the reader that this is the relavant code. We review some recent discussion of best practice for reproducibility in various professional organisations and journals, and discuss the current reward structure for publishing code in robotics, along with some ideas for improvement.