A Large-Scale Study on Developer Engagement and Expertise in Configurable Software System Projects

TL;DR

This large-scale study reveals that a small group of developers predominantly manage variable code in CSS projects, and current expertise metrics are insufficient for identifying these specialists, highlighting the need for improved assessment tools.

Contribution

It provides the first comprehensive analysis of developer engagement with variable code in CSS projects and evaluates the effectiveness of existing expertise metrics.

Findings

83% of variable code maintained by 17% of developers

Conventional expertise metrics achieve ~55% precision and 50% recall

Most developers (59%) do not modify variable code

Abstract

Modern systems operate in multiple contexts making variability a fundamental aspect of Configurable Software Systems (CSSs). Variability, implemented via pre-processor directives (e.g., #ifdef blocks) interleaved with other code and spread across files, complicates maintenance and increases error risk. Despite its importance, little is known about how variable code is distributed among developers or whether conventional expertise metrics adequately capture variable code proficiency. This study investigates developers' engagement with variable versus mandatory code, the concentration of variable code workload, and the effectiveness of expertise metrics in CSS projects. We mined repositories of 25 CSS projects, analyzing 450,255 commits from 9,678 developers. Results show that 59% of developers never modified variable code, while about 17% were responsible for developing and maintaining 83% of it. This indicates a high concentration of variable code expertise among a few developers, suggesting that task assignments should prioritize these specialists. Moreover, conventional expertise metrics performed poorly--achieving only around 55% precision and 50% recall in identifying developers engaged with variable code. Our findings highlight an unbalanced distribution of variable code responsibilities and underscore the need to refine expertise metrics to better support task assignments in CSS projects, thereby promoting a more equitable workload distribution.