Building Confidence in Scientific Computing Software Via Assurance Cases

TL;DR

This paper advocates for using assurance cases to improve the reliability and certification of Scientific Computing Software, demonstrated through a correctness case for a medical imaging application, highlighting benefits and current documentation issues.

Contribution

It introduces the application of assurance cases to Scientific Computing Software, illustrating their value through a detailed correctness case for a brain activity analysis tool.

Findings

Assurance cases can improve SCS certification.

Current documentation for SCS often has ambiguities.

Using assurance cases can identify critical software concerns.

Abstract

Assurance cases provide an organized and explicit argument for correctness. They can dramatically improve the certification of Scientific Computing Software (SCS). Assurance cases have already been effectively used for safety cases for real time systems. Their advantages for SCS include engaging domain experts, producing only necessary documentation, and providing evidence that can be verified/replicated. This paper illustrates assurance cases for SCS through the correctness case for 3dfim+, an existing Medical Imaging Application (MIA) for analyzing activity in the brain. This example was partly chosen because of recent concerns about the validity of fMRI (Functional Magnetic Resonance Imaging) studies. The example justifies the value of assurance cases for SCS, since the existing documentation is shown to have ambiguities and omissions, such as an incompletely defined ranking function and missing details on the coordinate system. A serious concern for 3dfim+ is identified: running the software does not produce any warning about the necessity of using data that matches the parametric statistical model employed for the correlation calculations. Raising the bar for SCS in general, and MIA in particular, is both feasible and necessary - when software impacts safety, an assurance case methodology (or an equivalently rigorous confidence building methodology) should be employed.