Test Behaviors, Not Methods! Detecting Tests Obsessed by Methods
Andre Hora, Andy Zaidman

TL;DR
This paper introduces a new test smell called 'Test Obsessed by Method' that identifies tests covering multiple behaviors via different paths in a single method, using runtime analysis, and demonstrates its presence in Python's standard library tests.
Contribution
The paper proposes a novel runtime analysis-based test smell detection method for identifying tests that verify multiple behaviors within a single production method.
Findings
44 tests exhibited the smell across 11 test suites
Smelly tests verify a median of two behaviors each
Detected tests could be split into 118 distinct tests
Abstract
Best testing practices state that tests should verify a single functionality or behavior of the system. Tests that verify multiple behaviors are harder to understand, lack focus, and are more coupled to the production code. An attempt to identify this issue is the test smell \emph{Eager Test}, which aims to capture tests that verify too much functionality based on the number of production method calls. Unfortunately, prior research suggests that counting production method calls is an inaccurate measure, as these calls do not reliably serve as a proxy for functionality. We envision a complementary solution based on runtime analysis: we hypothesize that some tests that verify multiple behaviors will likely cover multiple paths of the same production methods. Thus, we propose a novel test smell named \emph{Test Obsessed by Method}, a test method that covers multiple paths of a single…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Testing and Debugging Techniques · Software Engineering Research · Software System Performance and Reliability
