Tests as Maintainable Assets Via Auto-generated Spies: A case study involving the Scala collections library's Iterator trait

TL;DR

This paper demonstrates that using auto-generated test spies with mocking frameworks like Mockito significantly reduces test code complexity and enhances maintainability for Scala's Iterator trait, compared to manual spy implementations.

Contribution

It provides empirical evidence that auto-generated test spies improve code simplicity, readability, and style alignment in Scala testing practices.

Findings

Test code size reduced by over 70% with auto-generated spies

Cyclomatic complexity decreased significantly

Test code is more maintainable and stylistically aligned with Scala community

Abstract

In testing stateful abstractions, it is often necessary to record interactions, such as method invocations, and express assertions over these interactions. Following the Test Spy design pattern, we can reify such interactions programmatically through additional mutable state. Alternatively, a mocking framework, such as Mockito, can automatically generate test spies that allow us to record the interactions and express our expectations in a declarative domain-specific language. According to our study of the test code for Scala's Iterator trait, the latter approach can lead to a significant reduction of test code complexity in terms of metrics such as code size (in some cases over 70% smaller), cyclomatic complexity, and amount of additional mutable state required. In this tools paper, we argue that the resulting test code is not only more maintainable, readable, and intentional, but also a better stylistic match for the Scala community than manually implemented, explicitly stateful test spies.