Statistical considerations on safety analysis

TL;DR

This paper critically examines statistical methods used in safety analysis, highlighting weaknesses in common approaches and proposing the sign test as a more reliable alternative, with clarifications on Wilks' formula for multiple parameters.

Contribution

The authors identify flaws in the 0.95/0.95 methodology and introduce the sign test as a better alternative, also clarifying Wilks' formula for multiple parameters.

Findings

0.95/0.95 method can overestimate tolerance limits

Sign test offers a more reliable alternative for safety analysis

Correct form of Wilks' formula for multiple parameters clarified

Abstract

Alerting experience with a well-acknowledged safety analysis code initiated the authors to pay attention to safety issues of complex systems. Their first concern was the statistical characteristics of such a code. We point out a remarkable weakness of the so called 0.95/0.95 methodology: when repeating the search for the tolerance limit, we get a higher value with non-negligible probability. We propose the sign test as an alternative method. We point out the correct form of Wilks' formula when the number of parameters subjected to limitation is two or more.