Applying Deutsch's concept of good explanations to artificial intelligence and neuroscience -- an initial exploration

TL;DR

This paper explores how Deutsch's concept of hard-to-vary explanations can inform AI and neuroscience, emphasizing the importance of generating robust, extrapolative theories for advancing artificial general intelligence.

Contribution

It introduces a formal analysis of Deutsch's hard-to-vary principle, relating it to deep learning concepts and proposing a distinction between internal and external variability in models.

Findings

Hard-to-vary explanations are crucial for extrapolation and generalization.

Measuring internal variability via the Rashomon set and external variability via Kolmogorov complexity.

Identifies two brain systems: one perception-based and one capable of generating hard-to-vary explanations.

Abstract

Artificial intelligence has made great strides since the deep learning revolution, but AI systems still struggle to extrapolate outside of their training data and adapt to new situations. For inspiration we look to the domain of science, where scientists have been able to develop theories which show remarkable ability to extrapolate and sometimes predict the existence of phenomena which have never been observed before. According to David Deutsch, this type of extrapolation, which he calls "reach", is due to scientific theories being hard to vary. In this work we investigate Deutsch's hard-to-vary principle and how it relates to more formalized principles in deep learning such as the bias-variance trade-off and Occam's razor. We distinguish internal variability, how much a model/theory can be varied internally while still yielding the same predictions, with external variability, which is how much a model must be varied to accurately predict new, out-of-distribution data. We discuss how to measure internal variability using the size of the Rashomon set and how to measure external variability using Kolmogorov complexity. We explore what role hard-to-vary explanations play in intelligence by looking at the human brain and distinguish two learning systems in the brain. The first system operates similar to deep learning and likely underlies most of perception and motor control while the second is a more creative system capable of generating hard-to-vary explanations of the world. We argue that figuring out how replicate this second system, which is capable of generating hard-to-vary explanations, is a key challenge which needs to be solved in order to realize artificial general intelligence. We make contact with the framework of Popperian epistemology which rejects induction and asserts that knowledge generation is an evolutionary process which proceeds through conjecture and refutation.