Revisiting Method-Level Change Prediction: A Comparative Evaluation at Different Granularities

TL;DR

This study compares method-level and class-level change prediction techniques, revealing that method-level prediction can outperform class-level prediction at the method level, especially with limited maintenance effort, despite generally lower overall performance.

Contribution

The paper provides a comprehensive evaluation of method-level versus class-level change prediction across different granularities and effort considerations, clarifying their relative effectiveness.

Findings

Method-level prediction outperforms class-level at the method level with a median accuracy difference of 0.26.

Method-level prediction is more effective when minimal maintenance effort is acceptable.

Overall, method-level prediction has specific advantages despite lower general performance.

Abstract

To improve the efficiency of software maintenance, change prediction techniques have been proposed to predict frequently changing modules. Whereas existing techniques focus primarily on class-level prediction, method-level prediction allows for more direct identification of change locations. Method-level prediction can be useful, but it may also negatively affect prediction performance, leading to a trade-off. This makes it unclear which level of granularity users should select for their predictions. In this paper, we evaluated the performance of method-level change prediction compared with that of class-level prediction from three perspectives: direct comparison, method-level comparison, and maintenance effort-aware comparison. The results from 15 open source projects show that, although method-level prediction exhibited lower performance than class-level prediction in the direct comparison, method-level prediction outperformed class-level prediction when both were evaluated at method-level, leading to a median difference of 0.26 in accuracy. Furthermore, effort-aware comparison shows that method-level prediction performed significantly better when the acceptable maintenance effort is little.