When we can trust computers (and when we can't)

TL;DR

The paper discusses the capabilities and limitations of computational modeling across various scientific domains, emphasizing the importance of trust, validation, and the need for alternative methods in complex, less theoretically grounded fields.

Contribution

It provides a nuanced analysis of when computational methods are reliable and highlights the necessity of validation and alternative approaches in complex sciences.

Findings

Computational models are powerful in simple, theory-based domains.

Validation, verification, and data transparency are crucial for trust.

Complex systems in biology, medicine, and social sciences challenge computational reliability.

Abstract

With the relentless rise of computer power, there is a widespread expectation that computers can solve the most pressing problems of science, and even more besides. We explore the limits of computational modelling and conclude that, in the domains of science and engineering that are relatively simple and firmly grounded in theory, these methods are indeed powerful. Even so, the availability of code, data and documentation, along with a range of techniques for validation, verification and uncertainty quantification, are essential for building trust in computer generated findings. When it comes to complex systems in domains of science that are less firmly grounded in theory, notably biology and medicine, to say nothing of the social sciences and humanities, computers can create the illusion of objectivity, not least because the rise of big data and machine learning pose new challenges to reproducibility, while lacking true explanatory power. We also discuss important aspects of the natural world which cannot be solved by digital means. In the long-term, renewed emphasis on analogue methods will be necessary to temper the excessive faith currently placed in digital computation.