Experience-driven discovery of planning strategies

TL;DR

This paper explores how humans discover new planning strategies through metacognitive reinforcement learning, introducing models that better explain human strategy discovery but still lag in discovery speed.

Contribution

It proposes that metacognitive reinforcement learning underpins the discovery of new planning strategies and provides models that outperform alternative mechanisms in explaining this process.

Findings

Models demonstrate capability for strategy discovery.

Models better explain human strategy discovery.

Humans discover strategies faster than models.

Abstract

One explanation for how people can plan efficiently despite limited cognitive resources is that we possess a set of adaptive planning strategies and know when and how to use them. But how are these strategies acquired? While previous research has studied how individuals learn to choose among existing strategies, little is known about the process of forming new planning strategies. In this work, we propose that new planning strategies are discovered through metacognitive reinforcement learning. To test this, we designed a novel experiment to investigate the discovery of new planning strategies. We then present metacognitive reinforcement learning models and demonstrate their capability for strategy discovery as well as show that they provide a better explanation of human strategy discovery than alternative learning mechanisms. However, when fitted to human data, these models exhibit a slower discovery rate than humans, leaving room for improvement.