Assessing requirements engineering and software test alignment -- Five case studies

TL;DR

This paper introduces REST-bench, an assessment tool for evaluating and improving the alignment between requirements engineering and software testing in large, complex projects across various development methodologies.

Contribution

The paper presents REST-bench, a validated assessment method for analyzing RE and ST coordination, with iterative improvements based on five real-world case studies.

Findings

REST-bench effectively identified improvement opportunities.

Participants reported enhanced understanding of RE and ST coordination.

The method was applicable in diverse project environments.

Abstract

The development of large, software-intensive systems is a complex undertaking that we generally tackle by a divide and conquer strategy. Companies thereby face the challenge of coordinating individual aspects of software development, in particular between requirements engineering (RE) and software testing (ST). A lack of REST alignment can not only lead to wasted effort but also to defective software. However, before a company can improve the mechanisms of coordination they need to be understood first. With REST-bench we aim at providing an assessment tool that illustrates the coordination in software development projects and identify concrete improvement opportunities. We have developed REST-bench on the sound fundamentals of a taxonomy on REST alignment methods and validated the method in five case studies. Following the principles of technical action research, we collaborated with five companies, applying REST-bench and iteratively improving the method based on the lessons we learned. We applied REST-bench both in Agile and plan-driven environments, in projects lasting from weeks to years, and staffed as large as 1000 employees. The improvement opportunities we identified and the feedback we received indicate that the assessment was effective and efficient. Furthermore, participants confirmed that their understanding on the coordination between RE and ST improved.