Why Research on Test-Driven Development is Inconclusive?

TL;DR

This paper investigates why research on Test-Driven Development (TDD) yields inconclusive results by analyzing recent studies and identifying factors affecting outcomes, aiming to improve future research reliability and practical decision-making.

Contribution

It identifies five key factors influencing TDD research outcomes, providing insights to enhance study design and applicability in industrial contexts.

Findings

Five categories of factors impact TDD study results

Analysis of recent TDD research reveals methodological inconsistencies

Guidelines for more reliable TDD research design

Abstract

[Background] Recent investigations into the effects of Test-Driven Development (TDD) have been contradictory and inconclusive. This hinders development teams to use research results as the basis for deciding whether and how to apply TDD. [Aim] To support researchers when designing a new study and to increase the applicability of TDD research in the decision-making process in the industrial context, we aim at identifying the reasons behind the inconclusive research results in TDD. [Method] We studied the state of the art in TDD research published in top venues in the past decade, and analyzed the way these studies were set up. [Results] We identified five categories of factors that directly impact the outcome of studies on TDD. [Conclusions] This work can help researchers to conduct more reliable studies, and inform practitioners of risks they need to consider when consulting research on TDD.