On the Interplay between Non-Functional Requirements and Builds on Continuous Integration

TL;DR

This paper investigates how non-functional requirements influence build failures in continuous integration, revealing significant differences and proposing tools to improve CI efficiency and reduce delays, especially for Ruby and Java projects.

Contribution

It provides an empirical analysis of the relationship between NFRs and build statuses across 1,283 projects, highlighting potential improvements in CI processes.

Findings

Significant differences in build statuses related to NFRs.

Bimodal distribution of time to fix broken builds.

Planned maintenance could reduce delays for Ruby and Java.

Abstract

Continuous Integration (CI) implies that a whole developer team works together on the mainline of a software project. CI systems automate the builds of a software. Sometimes a developer checks in code, which breaks the build. A broken build might not be a problem by itself, but it has the potential to disrupt co-workers, hence it affects the performance of the team. In this study, we investigate the interplay between nonfunctional requirements (NFRs) and builds statuses from 1,283 software projects. We found significant differences among NFRs related-builds statuses. Thus, tools can be proposed to improve CI with focus on new ways to prevent failures into CI, specially for efficiency and usability related builds. Also, the time required to put a broken build back on track indicates a bimodal distribution along all NFRs, with higher peaks within a day and lower peaks in six weeks. Our results suggest that more planned schedule for maintainability for Ruby, and for functionality and reliability for Java would decrease delays related to broken builds.