Model-free, Model-based, and General Intelligence

TL;DR

This paper reviews the evolution of AI from hand-crafted programs to learning-based systems, highlighting the gap between model-free and model-based approaches and emphasizing the need for integrating these paradigms for robust, general intelligence.

Contribution

It provides a comprehensive review of AI development, analyzing the parallels with human cognition and discussing the critical gap between model-free and model-based methods.

Findings

Model-free learners are fast but opaque and inflexible.

Model-based solvers are slow but transparent and flexible.

Bridging the gap between these approaches is essential for robust, general AI.

Abstract

During the 60s and 70s, AI researchers explored intuitions about intelligence by writing programs that displayed intelligent behavior. Many good ideas came out from this work but programs written by hand were not robust or general. After the 80s, research increasingly shifted to the development of learners capable of inferring behavior and functions from experience and data, and solvers capable of tackling well-defined but intractable models like SAT, classical planning, Bayesian networks, and POMDPs. The learning approach has achieved considerable success but results in black boxes that do not have the flexibility, transparency, and generality of their model-based counterparts. Model-based approaches, on the other hand, require models and scalable algorithms. Model-free learners and model-based solvers have close parallels with Systems 1 and 2 in current theories of the human mind: the first, a fast, opaque, and inflexible intuitive mind; the second, a slow, transparent, and flexible analytical mind. In this paper, I review developments in AI and draw on these theories to discuss the gap between model-free learners and model-based solvers, a gap that needs to be bridged in order to have intelligent systems that are robust and general.