Detecting Continuous Integration Skip : A Reinforcement Learning-based Approach

TL;DR

This paper introduces a reinforcement learning-based method to accurately identify commits that can be skipped in Continuous Integration, reducing unnecessary resource usage in large, dependency-rich projects.

Contribution

It presents a novel deep reinforcement learning approach to build an optimal decision tree for CI skip detection, addressing data imbalance and outperforming existing methods.

Findings

Superior accuracy over state-of-the-art methods

Effective in diverse open-source projects

Reduces unnecessary CI resource consumption

Abstract

The software industry is experiencing a surge in the adoption of Continuous Integration (CI) practices, both in commercial and open-source environments. CI practices facilitate the seamless integration of code changes by employing automated building and testing processes. Some frameworks, such as Travis CI and GitHub Actions have significantly contributed to simplifying and enhancing the CI process, rendering it more accessible and efficient for development teams. Despite the availability these CI tools , developers continue to encounter difficulties in accurately flagging commits as either suitable for CI execution or as candidates for skipping especially for large projects with many dependencies. Inaccurate flagging of commits can lead to resource-intensive test and build processes, as even minor commits may inadvertently trigger the Continuous Integration process. The problem of detecting CI-skip commits, can be modeled as binary classification task where we decide to either build a commit or to skip it. This study proposes a novel solution that leverages Deep Reinforcement Learning techniques to construct an optimal Decision Tree classifier that addresses the imbalanced nature of the data. We evaluate our solution by running a within and a cross project validation benchmark on diverse range of Open-Source projects hosted on GitHub which showcased superior results when compared with existing state-of-the-art methods.