How are We Detecting Inconsistent Method Names? An Empirical Study from Code Review Perspective

TL;DR

This paper investigates how effectively current tools detect inconsistent method names by analyzing a new benchmark derived from code reviews, revealing significant performance gaps and biases in existing approaches.

Contribution

It introduces ReName4J, a review-based benchmark for method name inconsistency detection, and provides an empirical analysis of state-of-the-art techniques' limitations.

Findings

Existing techniques underperform on review-based benchmark

Potential biases affect evaluation of name inconsistency detection methods

Review-driven analysis offers new insights into method naming issues

Abstract

Proper naming of methods can make program code easier to understand, and thus enhance software maintainability. Yet, developers may use inconsistent names due to poor communication or a lack of familiarity with conventions within the software development lifecycle. To address this issue, much research effort has been invested into building automatic tools that can check for method name inconsistency and recommend consistent names. However, existing datasets generally do not provide precise details about why a method name was deemed improper and required to be changed. Such information can give useful hints on how to improve the recommendation of adequate method names. Accordingly, we construct a sample method-naming benchmark, ReName4J, by matching name changes with code reviews. We then present an empirical study on how state-of-the-art techniques perform in detecting or recommending consistent and inconsistent method names based on ReName4J. The main purpose of the study is to reveal a different perspective based on reviewed names rather than proposing a complete benchmark. We find that the existing techniques underperform on our review-driven benchmark, both in inconsistent checking and the recommendation. We further identify potential biases in the evaluation of existing techniques, which future research should consider thoroughly.