Automatically Removing Unnecessary Stubbings from Test Suites

TL;DR

This paper introduces ARUS, an automated technique for identifying and removing unnecessary stubbings in test suites, thereby simplifying testing and improving reliability in evolving software projects.

Contribution

ARUS automates the detection and removal of unnecessary stubbings in test suites, reducing manual effort and potential errors in software testing.

Findings

Resolves 98.6% of unnecessary stubbing cases in tested projects.

Negligible time cost for executing the technique.

Limited increase in code complexity after applying ARUS.

Abstract

Most modern software systems are characterized by a high number of components whose interactions can affect and complicate testing activities. During testing, developers can account for the interactions by isolating the code under test using test doubles and stubbings. During the evolution of a test suite, stubbings might become unnecessary, and developers should remove unnecessary stubbings, as their definitions can introduce unreliable test results in future versions of the test suite. Unfortunately, removing unnecessary stubbings is still a manual task that can be complex and time-consuming. To help developers in this task, we propose ARUS, a technique to automatically remove unnecessary stubbings from test suites. Given a software project and its test suite, the technique executes the tests to identify unnecessary stubbings and then removes them using different approaches based on the characteristics of the stubbings. We performed an empirical evaluation based on 128 Java projects that use Mockito for stubbing and contain 280 stubbing definitions that lead to 1,529 unnecessary stubbings. Overall, our technique provides a solution for 276 of the definitions (98.6% resolution rate), ARUS' time cost is negligible, and, on average, the technique's changes introduce a limited increase in code complexity. We submitted ARUS' changes to the projects through pull requests and 83 resolutions are already merged.