In industrial embedded software, are some compilation errors easier to localize and fix than others?

TL;DR

This study investigates whether certain compilation errors in industrial embedded software are easier to localize and fix, analyzing a large dataset of build errors to identify patterns and factors affecting error resolution.

Contribution

The paper introduces 'Shadow Job', a CI diagnostics tool, and provides an empirical analysis of compilation error types, their frequency, and localization difficulty in industrial embedded systems.

Findings

Most common errors account for 89% of issues

Resolution time, size, and distance are independent factors

Insights into human effort for fixing industrial compilation errors

Abstract

Industrial embedded systems often require specialized hardware. However, software engineers have access to such domain-specific hardware only at the continuous integration (CI) stage and have to use simulated hardware otherwise. This results in a higher proportion of compilation errors at the CI stage than in other types of systems, warranting a deeper study. To this end, we create a CI diagnostics solution called ``Shadow Job'' that analyzes our industrial CI system. We collected over 40000 builds from 4 projects from the product source code and categorized the compilation errors into 14 error types, showing that the five most common ones comprise 89 % of all compilation errors. Additionally, we analyze the resolution time, size, and distance for each error type, to see if different types of compilation errors are easier to localize or repair than others. Our results show that the resolution time, size, and distance are independent of each other. Our research also provides insights into the human effort required to fix the most common industrial compilation errors. We also identify the most promising directions for future research on fault localization.