Improving Merge Pipeline Throughput in Continuous Integration via Pull Request Prioritization

TL;DR

This paper presents a method to improve merge pipeline throughput in continuous integration by prioritizing pull requests based on build success likelihood, using historical data and metadata, which outperforms traditional ordering strategies.

Contribution

The paper introduces a build prediction-based PR prioritization method that enhances merge pipeline throughput without relying on specific build systems.

Findings

Predictive PR ordering outperforms FIFO and non-learning strategies.

The approach is system-agnostic and easily integrable.

Significant throughput improvements demonstrated on large-scale projects.

Abstract

Integrating changes into large monolithic software repositories is a critical step in modern software development that substantially impacts the speed of feature delivery, the stability of the codebase, and the overall productivity of development teams. To ensure the stability of the main branch, many organizations use merge pipelines that test software versions before the changes are permanently integrated. However, the load on merge pipelines is often so high that they become bottlenecks, despite the use of parallelization. Existing optimizations frequently rely on specific build systems, limiting their generalizability and applicability. In this paper we propose to optimize the order of PRs in merge pipelines using practical build predictions utilizing only historical build data, PR metadata, and contextual information to estimate the likelihood of successful builds in the merge pipeline. By dynamically prioritizing likely passing PRs during peak hours, this approach maximizes throughput when it matters most. Experiments conducted on a real-world, large-scale project demonstrate that predictive ordering significantly outperforms traditional first-in-first-out (FIFO), as well as non-learning-based ordering strategies. Unlike alternative optimizations, this approach is agnostic to the underlying build system and thus easily integrable into existing automated merge pipelines.