Individual differences in the cognitive mechanisms of planning strategy discovery

TL;DR

This study explores how individual differences and cognitive mechanisms influence the discovery of planning strategies in humans, enhancing existing models with factors like effort valuation and pseudo-rewards to better match human performance.

Contribution

The paper introduces additional cognitive mechanisms into metacognitive reinforcement learning models to better account for individual differences in planning strategy discovery.

Findings

A significant proportion of participants used at least one new mechanism.

Mechanisms like pseudo-rewards and effort valuation facilitated strategy discovery.

Models with these mechanisms still did not fully match human performance.

Abstract

People employ efficient planning strategies. But how are these strategies acquired? Previous research suggests that people can discover new planning strategies through learning from reinforcements, a process known as metacognitive reinforcement learning (MCRL). While prior work has shown that MCRL models can learn new planning strategies and explain more participants' experience-driven discovery better than alternative mechanisms, it also revealed significant individual differences in metacognitive learning. Furthermore, when fitted to human data, these models exhibit a slower rate of strategy discovery than humans. In this study, we investigate whether incorporating cognitive mechanisms that might facilitate human strategy discovery can bring models of MCRL closer to human performance. Specifically, we consider intrinsically generated metacognitive pseudo-rewards, subjective effort valuation, and termination deliberation. Analysis of planning task data shows that a larger proportion of participants used at least one of these mechanisms, with significant individual differences in their usage and varying impacts on strategy discovery. Metacognitive pseudo-rewards, subjective effort valuation, and learning the value of acting without further planning were found to facilitate strategy discovery. While these enhancements provided valuable insights into individual differences and the effect of these mechanisms on strategy discovery, they did not fully close the gap between model and human performance, prompting further exploration of additional factors that people might use to discover new planning strategies.