Developers perception on the severity of test smells: an empirical study

TL;DR

This empirical study investigates developers' perceptions of test smell severity in unit tests, revealing that while generally viewed as low severity, test smells can negatively affect maintainability and evolution of test code.

Contribution

It provides insights into developers' perceptions of test smell severity and highlights the potential benefits of detecting and removing test smells for project quality.

Findings

Most developers see test smells as low severity.

Test smells may negatively impact maintainability and evolution.

Removing test smells can benefit the project.

Abstract

Unit testing is an essential component of the software development life-cycle. A developer could easily and quickly catch and fix software faults introduced in the source code by creating and running unit tests. Despite their importance, unit tests are subject to bad design or implementation decisions, the so-called test smells. These might decrease software systems quality from various aspects, making it harder to understand, more complex to maintain, and more prone to errors and bugs. Many studies discuss the likely effects of test smells on test code. However, there is a lack of studies that capture developers perceptions of such issues. This study empirically analyzes how developers perceive the severity of test smells in the test code they develop. Severity refers to the degree to how a test smell may negatively impact the test code. We selected six open-source software projects from GitHub and interviewed their developers to understand whether and how the test smells affected the test code. Although most of the interviewed developers considered the test smells as having a low severity to their code, they indicated that test smells might negatively impact the project, particularly in test code maintainability and evolution. Also, detecting and removing test smells from the test code may be positive for the project.