Investigating the Impact of Continuous Integration Practices on the Productivity and Quality of Open-Source Projects

TL;DR

This study evaluates how five specific Continuous Integration sub-practices influence the productivity and quality of open-source projects on GitHub, revealing that better adherence correlates with increased activity and fewer bugs.

Contribution

It uniquely analyzes the impact of specific CI sub-practices on open-source project productivity and quality, beyond just adopting CI tools.

Findings

Build and Commit Activity correlate with merged pull requests.

Build Activity, Build Health, and Time to Fix Broken Builds relate to fewer bug issues.

Projects with better CI practices face fewer CI-related problems.

Abstract

Background: Much research has been conducted to investigate the impact of Continuous Integration (CI) on the productivity and quality of open-source projects. Most of studies have analyzed the impact of adopting a CI server service (e.g, Travis-CI) but did not analyze CI sub-practices. Aims: We aim to evaluate the impact of five CI sub-practices with respect to the productivity and quality of GitHub open-source projects. Method: We collect CI sub-practices of 90 relevant open-source projects for a period of 2 years. We use regression models to analyze whether projects upholding the CI sub-practices are more productive and/or generate fewer bugs. We also perform a qualitative document analysis to understand whether CI best practices are related to a higher quality of projects. Results: Our findings reveal a correlation between the Build Activity and Commit Activity sub-practices and the number of merged pull requests. We also observe a correlation between the Build Activity, Build Health and Time to Fix Broken Builds sub-practices and number of bug-related issues. The qualitative analysis reveals that projects with the best values for CI sub-practices face fewer CI-related problems compared to projects that exhibit the worst values for CI sub-practices. Conclusions: We recommend that projects should strive to uphold the several CI sub-practices as they can impact in the productivity and quality of projects.