Induction, Popper, and machine learning

TL;DR

This paper critiques the induction-based view of scientific and AI development, advocating for a universal Darwinian framework as a more accurate foundation for understanding AI algorithms and their evolution.

Contribution

It contrasts induction with Darwinian evolution, proposing that AI algorithms are better understood through universal Darwinism rather than induction-based frameworks.

Findings

Most AI algorithms operate as evolutionary trial-and-error processes.

Universal Darwinian framework offers a superior foundation for understanding AI development.

Development of AI algorithms can be viewed as a form of universal Darwinism.

Abstract

Francis Bacon popularized the idea that science is based on a process of induction by which repeated observations are, in some unspecified way, generalized to theories based on the assumption that the future resembles the past. This idea was criticized by Hume and others as untenable leading to the famous problem of induction. It wasn't until the work of Karl Popper that this problem was solved, by demonstrating that induction is not the basis for science and that the development of scientific knowledge is instead based on the same principles as biological evolution. Today, machine learning is also taught as being rooted in induction from big data. Solomonoff induction implemented in an idealized Bayesian agent (Hutter's AIXI) is widely discussed and touted as a framework for understanding AI algorithms, even though real-world attempts to implement something like AIXI immediately encounter fatal problems. In this paper, we contrast frameworks based on induction with Donald T. Campbell's universal Darwinism. We show that most AI algorithms in use today can be understood as using an evolutionary trial and error process searching over a solution space. In this work we argue that a universal Darwinian framework provides a better foundation for understanding AI systems. Moreover, at a more meta level the process of development of all AI algorithms can be understood under the framework of universal Darwinism.