Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD

TL;DR

This study evaluates whether Test-Driven Development (TDD) consistently improves external software quality across different industrial settings, finding that alternative approaches like Iterative-Test Last (ITL) often outperform TDD, especially with experienced testers.

Contribution

The paper provides empirical evidence on TDD's performance across companies and premises, highlighting participant characteristics and showing ITL's superior performance in most cases.

Findings

ITL outperforms TDD in three out of four premises

ITL outperforms TDD in both companies

Experience with testing tools amplifies performance differences

Abstract

Test-Driven Development (TDD) has been claimed to increase external software quality. However, the extent to which TDD increases external quality has been seldom studied in industrial experiments. We conduct four industrial experiments in two different companies to evaluate the performance of TDD on external quality. We study whether the performance of TDD holds across premises within the same company and across companies. We identify participant-level characteristics impacting results. Iterative-Test Last (ITL), the reverse approach of TDD, outperforms TDD in three out of four premises. ITL outperforms TDD in both companies. The larger the experience with unit testing and testing tools, the larger the difference in performance between ITL and TDD (in favour of ITL). Technological environment (i.e., programming language and testing tool) seems not to impact results. Evaluating participant-level characteristics impacting results in industrial experiments may ease the understanding of the performance of TDD in realistic settings.