Interleaving Fast and Slow Decision Making

TL;DR

This paper introduces a framework for interleaving fast and slow decision-making systems, including a new overseeing System 0, and demonstrates its effectiveness in a modified Pac-Man game with reinforcement learning and Monte Carlo tree search.

Contribution

The paper proposes a novel three-system framework with a supervisory System 0 to effectively combine fast and slow decision-making in AI agents.

Findings

System 0 improves overall performance over single-system approaches.

Certain strategies for System 0 enable better decision-making in dynamic environments.

Interleaving decision systems outperforms arbitrary switching methods.

Abstract

The "Thinking, Fast and Slow" paradigm of Kahneman proposes that we use two different styles of thinking -- a fast and intuitive System 1 for certain tasks, along with a slower but more analytical System 2 for others. While the idea of using this two-system style of thinking is gaining popularity in AI and robotics, our work considers how to interleave the two styles of decision-making, i.e., how System 1 and System 2 should be used together. For this, we propose a novel and general framework which includes a new System 0 to oversee Systems 1 and 2. At every point when a decision needs to be made, System 0 evaluates the situation and quickly hands over the decision-making process to either System 1 or System 2. We evaluate such a framework on a modified version of the classic Pac-Man game, with an already-trained RL algorithm for System 1, a Monte-Carlo tree search for System 2, and several different possible strategies for System 0. As expected, arbitrary switches between Systems 1 and 2 do not work, but certain strategies do well. With System 0, an agent is able to perform better than one that uses only System 1 or System 2.