Introducing Automated Regression Testing in Open Source Projects

TL;DR

This paper explores the process and benefits of implementing automated regression testing in open source projects through a long-term case study, highlighting practical strategies and social dynamics involved.

Contribution

It presents a detailed case study demonstrating how to successfully introduce automated testing into open source projects and identifies key strategies and social factors influencing adoption.

Findings

Testing improves project quality and communication

Automated testing is feasible for external innovators

Signaling encourages community engagement

Abstract

To learn how to introduce automated regression testing to existing medium scale Open Source projects, a long-term field experiment was performed with the Open Source project FreeCol. Results indicate that (1) introducing testing is both beneficial for the project and feasible for an outside innovator, (2) testing can enhance communication between developers, (3) signaling is important for engaging the project participants to fill a newly vacant position left by a withdrawal of the innovator. Five prescriptive strategies are extracted for the innovator and two conjectures offered about the ability of an Open Source project to learn about innovations.