Evaluating Heuristics for Iterative Impact Analysis

TL;DR

This paper evaluates various heuristics for iterative impact analysis in software engineering, finding that while IIA improves recall over other techniques, the studied heuristics do not outperform random inspection in guiding impact estimation.

Contribution

The study systematically assesses propagation and termination heuristics for IIA, revealing their limited effectiveness in improving impact estimation accuracy.

Findings

IIA achieves better recall than other impact analysis methods.

The investigated heuristics do not outperform random inspection in impact estimation.

Heuristics studied do not significantly enhance IIA performance.

Abstract

Iterative impact analysis (IIA) is a process that allows developers to estimate the impacted units of a software change. Starting from a single impacted unit, the developers inspect its interacting units via program dependencies to identify the ones that are also impacted, and this process continues iteratively. Experience has shown that developers often miss impacted units and inspect many irrelevant units. In this work, we study propagation heuristics that guide developers to find the actual impacted units and termination heuristics that help to decide whether the estimated impact is complete. The roles of these two kinds of heuristics are complementary and affect both the precision and recall when used during IIA. We investigated several propagation heuristics adapted from previously published papers and combined them with a practical termination heuristic. We developed a reenactment process that simulates the actions of developers who use those heuristics during IIA, and we assessed their performance. The software changes for our reenactment were mined from the repositories of open source projects. We found that IIA provides better recall than the other known impact analysis techniques. However the IIA with the propagation heuristics that we investigated does not supersede IIA combined with a random inspection, and hence these heuristics do not help the IIA.