Automated and manual testing as part of the research software development process of RCE

TL;DR

This paper discusses a balanced approach combining automated and manual testing processes in the development of the research software framework RCE, aiming to improve reliability while managing limited resources.

Contribution

It introduces specific testing processes for RCE that integrate automation and manual efforts to enhance software quality in research environments.

Findings

Reduced testing overhead through combined processes

Early issue detection in software development

Effective integration with development and release workflows

Abstract

Research software is often developed by individual researchers or small teams in parallel to their research work. The more people and research projects rely on the software in question, the more important it is that software updates implement new features correctly and do not introduce regressions. Thus, developers of research software must balance their limited resources between implementing new features and thoroughly testing any code changes. We present the processes we use for developing the distributed integration framework RCE at DLR. These processes aim to strike a balance between automation and manual testing, reducing the testing overhead while addressing issues as early as possible. We furthermore briefly describe how these testing processes integrate with the surrounding processes for development and releasing.