Testing as an Investment

TL;DR

This paper applies an investment model to evaluate software testing strategies, providing a financial perspective on testing returns over different time horizons to aid decision-making.

Contribution

It introduces the Nelson-Siegel investment model to analyze and forecast testing process returns, offering a novel financial viewpoint in software testing evaluation.

Findings

Statement-coverage yields the best long-term returns.

Testing benefits vary with program size and fault count.

Different testing criteria have distinct investment profiles.

Abstract

Software testing is an expensive and important task. Plenty of researches and industrial efforts have been invested on improving software testing techniques, including criteria, tools, etc. These studies can provide guidelines to select suitable test techniques for software engineers. However, in some engineering projects, business issues may be more important than technical ones, hence we need to lobby non-technical members to support our decisions. In this paper, a well-known investment model, Nelson-Siegel model, is introduced to evaluate and forecast the processes of testing with different testing criteria. Through this model, we provide a new perspective to understand short-term, medium-term, and long-term returns of investments throughout the process of testing. A preliminary experiment is conducted to investigate three testing criteria from the viewpoint of investments. The results show that statement-coverage criterion performs best in gaining long-term yields; the short-term and medium-term yields of testing depend on the scale of programs and the number of faults they contain.