Decisions in Continuous Integration and Delivery: An Exploratory Study

TL;DR

This study explores decision-making in CI/CD by analyzing issues from an open-source project, revealing that most decisions relate to functional requirements and architecture, thus providing insights for practitioners and researchers.

Contribution

It introduces a decision classification framework for CI/CD by analyzing real-world issues, filling a gap in understanding decision patterns in CI/CD practices.

Findings

Majority of decisions are about functional requirements (67.6%)

Architecture decisions constitute 11.1% of the decisions

Provides a decision classification reference for CI/CD practices

Abstract

In recent years, Continuous Integration (CI) and Continuous Delivery (CD) has been heatedly discussed and widely used in part or all of the software development life cycle as the practices and pipeline to deliver software products in an efficient way. There are many tools, such as Travis CI, that offer various features to support the CI/CD pipeline, but there is a lack of understanding about what decisions are frequently made in CI/CD. In this work, we explored one popular open-source project on GitHub, Budibase, to provide insights on the types of decisions made in CI/CD from a practitioners' perspective. We first explored the GitHub Trending page, conducted a pilot repository extraction, and identified the Budibase repository as the case for our study. We then crawled all the closed issues from the repository and got 1,168 closed issues. Irrelevant issues were filtered out based on certain criteria, and 370 candidate issues that contain decisions were obtained for data extraction. We analyzed the issues using a hybrid approach combining pre-defined types and the Constant Comparison method to get the categories of decisions. The results show that the major type of decisions in the Budibase closed issues is Functional Requirement Decision (67.6%), followed by Architecture Decision (11.1%). Our findings encourage developers to put more effort on the issues and making decisions related to CI/CD, and provide researchers with a reference of decision classification made in CI/CD.