DDTS: A Practical System Testing Framework for Scientific Software

TL;DR

DDTS is a practical, portable testing framework designed to simplify and improve testing processes for scientific software, especially on HPC platforms, by making configuration easy and supporting parallelism.

Contribution

It introduces DDTS, a flexible testing framework tailored for scientific software, addressing challenges like complexity, portability, and integration with HPC environments.

Findings

Used for automated regression testing in scientific projects

Supports parallel testing to improve efficiency

Eases adoption of rigorous testing practices

Abstract

Many scientific-software projects test their codes inadequately, or not at all. Despite its well-known benefits, adopting routine testing is often not easy. Development teams may have doubts about establishing effective test procedures, writing test software, or handling the ever-growing complexity of test cases. They may need to run (and test) on restrictive HPC platforms. They almost certainly face time and budget pressures that can keep testing languishing near the bottom of their to-do lists. This paper presents DDTS, a framework for building test suite applications, designed to fit scientific-software projects' requirements. DDTS aims to simplify introduction of rigorous testing, and to ease growing pains as needs mature. It decomposes the testing problem into practical, intuitive phases; makes configuration and extension easy; is portable and suitable to HPC platforms; and exploits parallelism. DDTS is currently used for automated regression and developer pre-commit testing for several scientific-software projects with disparate testing requirements.