The Cost of Downgrading Build Systems: A Case Study of Kubernetes

TL;DR

This case study of Kubernetes reveals that downgrading from an artifact-based build system like Bazel to a language-specific tool like Go Build results in significant performance trade-offs, including increased build times and memory usage, impacting CI costs.

Contribution

The paper provides an empirical analysis of the performance implications of build system downgrades, highlighting the costs and trade-offs involved in switching from Bazel to Go Build in large projects.

Findings

Bazel builds are faster than Go Build for full and incremental builds.

Downgrading increases memory consumption across projects.

Build time penalties decrease over time, but memory costs remain high.

Abstract

Since developers invoke the build system frequently, its performance can impact productivity. Modern artifact-based build tools accelerate builds, yet prior work shows that teams may abandon them for alternatives that are easier to maintain. While prior work shows why downgrades are performed, the implications of downgrades remain largely unexplored. In this paper, we describe a case study of the Kubernetes project, focusing on its downgrade from an artifact-based build tool (Bazel) to a language-specific solution (Go Build). We reproduce and analyze the full and incremental builds of change sets during the downgrade period. On the one hand, we find that Bazel builds are faster than Go Build, completing full builds in 23.06-38.66 up to 75.19 impose a larger memory footprint than Go Build of 81.42-351.07 respectively. Bazel builds also impose a greater CPU load at parallelism settings above eight for full builds and above one for incremental builds. We estimate that downgrading from Bazel can increase CI resource costs by up to 76 explore whether our observations generalize by replicating our Kubernetes study on four other projects that also downgraded from Bazel to older build tools. We observe that while build time penalties decrease, Bazel consistently consumes more memory. We conclude that abandoning artifact-based build tools, despite perceived maintainability benefits, tends to incur considerable performance costs for large projects. Our observations may help stakeholders to balance trade-offs in build tool adoption