Off-the-shelf deep learning is not enough: parsimony, Bayes and causality

TL;DR

Deep learning has achieved remarkable success in AI but faces challenges in physical sciences due to issues with causality, requiring integration of Bayesian, physical constraints, and causal models for reliable scientific insights.

Contribution

The paper highlights the limitations of off-the-shelf deep learning in physical sciences and advocates for Bayesian, causal, and physics-informed approaches to improve scientific modeling.

Findings

Deep learning excels when causal links are known or stable.

Incorporating prior knowledge and physical constraints enhances reliability.

Causal models are essential to avoid misleading results in scientific applications.

Abstract

Deep neural networks ("deep learning") have emerged as a technology of choice to tackle problems in natural language processing, computer vision, speech recognition and gameplay, and in just a few years has led to superhuman level performance and ushered in a new wave of "AI." Buoyed by these successes, researchers in the physical sciences have made steady progress in incorporating deep learning into their respective domains. However, such adoption brings substantial challenges that need to be recognized and confronted. Here, we discuss both opportunities and roadblocks to implementation of deep learning within materials science, focusing on the relationship between correlative nature of machine learning and causal hypothesis driven nature of physical sciences. We argue that deep learning and AI are now well positioned to revolutionize fields where causal links are known, as is the case for applications in theory. When confounding factors are frozen or change only weakly, this leaves open the pathway for effective deep learning solutions in experimental domains. Similarly, these methods offer a pathway towards understanding the physics of real-world systems, either via deriving reduced representations, deducing algorithmic complexity, or recovering generative physical models. However, extending deep learning and "AI" for models with unclear causal relationship can produce misleading and potentially incorrect results. Here, we argue the broad adoption of Bayesian methods incorporating prior knowledge, development of DL solutions with incorporated physical constraints, and ultimately adoption of causal models, offers a path forward for fundamental and applied research. Most notably, while these advances can change the way science is carried out in ways we cannot imagine, machine learning is not going to substitute science any time soon.