Build Optimization: A Systematic Literature Review

TL;DR

This systematic review analyzes 97 studies on build optimization in Continuous Integration, highlighting techniques for reducing build durations and failures, with a focus on machine learning methods and publicly available datasets.

Contribution

It provides a comprehensive classification of existing build optimization techniques, datasets, and metrics, guiding future research and practice in CI build management.

Findings

Techniques for predicting build outcomes and durations

Methods for automating build failure repair

Identification of publicly available build datasets

Abstract

Continuous Integration (CI) consists of an automated build process involving continuous compilation, testing, and packaging of the software system. While CI comes up with several advantages related to quality and time to delivery, CI also presents several challenges addressed by a large body of research. To better understand the literature so as to help practitioners find solutions for their problems and guide future research, we conduct a systematic review of 97 studies on build optimization published between 2006 and 2024, which we summarized according to their goals, methodologies, used datasets, and leveraged metrics. The identified build optimization studies focus on two main challenges: (1) long build durations, and (2) build failures. To meet the first challenge, existing studies have developed a range of techniques, including predicting build outcome and duration, selective build execution, and build acceleration using caching or repairing performance smells. The causes of build failures have been the subject of several studies, leading to the development of techniques for predicting build script maintenance and automating repair. Recent studies have also focused on predicting flaky build failures caused by environmental issues. The majority of these techniques use machine learning algorithms and leverage build metrics, which we classify into five categories. Additionally, we identify eight publicly available build datasets for build optimization research.