Mandating Code Disclosure is Unnecessary -- Strict Model Verification Does Not Require Accessing Original Computer Code

TL;DR

This paper argues that model verification can be achieved without access to original code by using independent mental reconstruction from publication specifications, emphasizing human intelligence over code access.

Contribution

It introduces a verification approach that relies on human interpretation of publication text, reducing the need for public code disclosure.

Findings

Verification can be achieved without original code

Human interpretation can ensure model accuracy

Effort-directed feedback improves verification process

Abstract

Mandating public availability of computer code underlying computational simulation modeling research ends up doing a disservice to the cause of model verification when inconsistencies between the specifications in the publication text and specifications in the computer code go unchallenged. Conversely, a model is verified when an independent researcher undertakes the set of mental processing tasks necessary to convert natural language specifications in a publication text into computer code instructions that produce numerical or graphical outputs identical to the outputs found in the original publication. The effort towards obtaining convergence with the numerical or graphical outputs directs intensive consideration of the publication text. The original computer code has little role to play in determining the verification status - verified/ failed verification. An insight is obtained that skillful deployment of human intelligence is feasible when effort-directing feedback processes are in place to appropriately go around the human frailty of giving up in the absence of actionable feedback. This principle can be put to use to develop better organizational configurations in business, government and society.