What do I make of your Latinorum? Sensitivity auditing of mathematical modelling

TL;DR

This paper advocates for a structured approach called sensitivity auditing to improve the quality and transparency of mathematical models used in policy-making, emphasizing the treatment of uncertainty.

Contribution

It introduces seven rules for extending sensitivity analysis into sensitivity auditing, enhancing model credibility in policy contexts.

Findings

Seven rules for sensitivity auditing are proposed.

Illustrative examples demonstrate application of rules.

Enhanced transparency and reliability in policy models.

Abstract

In their book Uncertainty and Quality of Science for Policy (1990), Funtowicz and Ravetz argued the need to extend traditional methods and techniques of quality assurance in policy-related science. Since then, these ideas have been operationalized further and applied. Particularly relevant have been the recourse to extended peer review, to be intended as internal, across disciplines, as well as external, between practitioners and stakeholders, and the use of a new approach to qualify quantities: NUSAP (Numeral, Unit, Spread, Assessment, Pedigree). Here we describe how sensitivity analysis, mandated by existing guidelines as a good practice to use in conjunction to mathematical modelling, needs to be transformed and adapted to ensure quality in the treatment of uncertainty of science for policy. We thus provide seven rules to extend the use of sensitivity analysis (or how to apportion uncertainty in model based inference among input factors) in a process of sensitivity auditing of models used in a policy context. Each rule will be illustrated by examples.