Incrementalizing Production CodeQL Analyses

TL;DR

This paper investigates whether incremental static analysis can speed up production CodeQL analyses on GitHub pull requests by reusing previous results, demonstrating promising update times proportional to code change size.

Contribution

The study empirically validates the potential of incrementalization for CodeQL analyses and introduces a prototype incremental solver that exploits this approach.

Findings

Update times are proportional to code change size.

Incremental analysis can significantly reduce re-analysis time.

Prototype demonstrates feasibility despite limitations.

Abstract

Instead of repeatedly re-analyzing from scratch, an incremental static analysis only analyzes a codebase once completely, and then it updates the previous results based on the code changes. While this sounds promising to achieve speed-ups, the reality is that sophisticated static analyses typically employ features that can ruin incremental performance, such as inter-procedurality or context-sensitivity. In this study, we set out to explore whether incrementalization can help to achieve speed-ups for production CodeQL analyses that provide automated feedback on pull requests on GitHub. We first empirically validate the idea by measuring the potential for reuse on real-world codebases, and then we create a prototype incremental solver for CodeQL that exploits incrementality. We report on experimental results showing that we can indeed achieve update times proportional to the size of the code change, and we also discuss the limitations of our prototype.