A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last?

TL;DR

This study investigates how specific process characteristics of Test-Driven Development (TDD) influence software quality and productivity, revealing that granularity and uniformity are beneficial, while sequencing and refactoring effort have less clear effects.

Contribution

It introduces four process characteristics of TDD, analyzes their impact on quality and productivity, and challenges assumptions about the importance of test-first sequencing.

Findings

Granularity and uniformity positively impact quality and productivity.

Sequencing has no significant influence on outcomes.

Refactoring effort negatively affects quality and productivity.

Abstract

Background: Test-driven development (TDD) is a technique that repeats short coding cycles interleaved with testing. The developer first writes a unit test for the desired functionality, followed by the necessary production code, and refactors the code. Many empirical studies neglect unique process characteristics related to TDD iterative nature. Aim: We formulate four process characteristic: sequencing, granularity, uniformity, and refactoring effort. We investigate how these characteristics impact quality and productivity in TDD and related variations. Method: We analyzed 82 data points collected from 39 professionals, each capturing the process used while performing a specific development task. We built regression models to assess the impact of process characteristics on quality and productivity. Quality was measured by functional correctness. Result: Quality and productivity improvements were primarily positively associated with the granularity and uniformity. Sequencing, the order in which test and production code are written, had no important influence. Refactoring effort was negatively associated with both outcomes. We explain the unexpected negative correlation with quality by possible prevalence of mixed refactoring. Conclusion: The claimed benefits of TDD may not be due to its distinctive test-first dynamic, but rather due to the fact that TDD-like processes encourage fine-grained, steady steps that improve focus and flow.