Dear CAV, We Need to Talk About Reproducibility

TL;DR

This paper discusses the reproducibility crisis in CAV research and proposes an infrastructure to automate build, testing, and benchmarking to improve reproducibility and scientific progress.

Contribution

It introduces a specialized infrastructure to automate reproducibility tasks specifically tailored for the CAV research community.

Findings

Proposes an infrastructure for automating reproducibility tasks.

Addresses the reproducibility crisis in CAV research.

Aims to facilitate reliable re-implementation of research tools.

Abstract

How many times have you tried to re-implement a past CAV tool paper, and failed? Reliably reproducing published scientific discoveries has been acknowledged as a barrier to scientific progress for some time but there remains only a small subset of software available to support the specific needs of the research community (i.e. beyond generic tools such as source code repositories). In this paper we propose an infrastructure for enabling reproducibility in our community, by automating the build, unit testing and benchmarking of research software.