Automated Statistical Testing and Certification of a Reliable Model-Coupling Server for Scientific Computing

TL;DR

This paper introduces a statistical testing framework to verify and certify the reliability of a web service used for coupling scientific models, ensuring robustness in complex scientific computing applications.

Contribution

It presents a novel combination of specification and statistical testing methods for quality assurance of model-coupling web services in scientific computing.

Findings

Statistical testing uncovers issues missed by unit tests.

Reliability certification provides a quantitative robustness measure.

Approach enhances confidence in complex scientific simulations.

Abstract

Sequence-based specification and usage-driven statistical testing are designed for rigorous and cost-effective software development, offering a semi-formal approach to assessing the behavior of complex systems and interactions between various components. This approach is particularly valuable for scientific computing applications in which comprehensive tests are needed to prevent flawed results or conclusions. As scientific discovery becomes increasingly more complex, domain scientists couple multiple scientific computing models or simulations to solve intricate multiphysics and multiscale problems. These model-coupling applications use a hardwired coupling program or a flexible web service to link and combine different models. In this paper, we focus on the quality assurance of the more elastic web service via a combination of rigorous specification and testing methods. The application of statistical testing exposes problems ignored by pre-written unit tests and highlights areas in the code where failures might occur. We certify the model-coupling server controller with a derived reliability statistic, offering a quantitative measure to support a claim of its robustness.