Assessing the Feasibility of Selective Instrumentation for Runtime Code Coverage in Large C++ Game Engines

TL;DR

This paper presents a selective instrumentation method for large C++ game engines that reduces overhead and maintains test stability, enabling effective code coverage feedback during development.

Contribution

The paper introduces a novel selective instrumentation framework tailored for large C++ game engines, balancing coverage detail with minimal performance impact.

Findings

Over 2,000 commits instrumented with minimal build time increase

Frame rates maintained above 50% of baseline in worst-case scenarios

No automated test failures caused by the framework in production test suites

Abstract

Code coverage is a valuable guide for testing, but in AAA games the overhead of instrumentation conflicts with strict performance requirements and can destabilize automated tests. We propose and assess a selective instrumentation approach tailored to large game engines written in \texttt{C++}, which reduces the scope of instrumentation while preserving relevant coverage data to developer commits. Our framework integrates into an industrial game testing pipeline, enabling developers to receive immediate coverage feedback on tests run against their changes. The compilation overhead of our approach is minimal, allowing instrumentation of over 2,000 commits before doubling build time. In performance evaluations, even the worst-case scenario maintains frame rates above 50\% of the non-instrumented baseline. Across two production test suites maintained by our industry partner, our framework caused no automated test failures, avoiding the instability observed under full instrumentation. Our work shows that commit-level or build-level coverage of large \texttt{C++} game engines can be achieved with minimal overhead and without compromising test stability.