Primate-like perceptual decision making emerges through deep recurrent reinforcement learning

TL;DR

This study demonstrates that deep recurrent reinforcement learning models can develop primate-like perceptual decision-making abilities, including speed-accuracy trade-offs and flexible decision changes, supporting theories on the evolution of such mechanisms.

Contribution

The paper shows that reinforcement learning in deep recurrent networks can produce decision-making behaviors similar to primates, providing insights into the emergence of these mechanisms.

Findings

Networks exhibit speed-accuracy trade-offs.

Networks can change decisions based on new information.

Internal dynamics resemble primate neurophysiology.

Abstract

Progress has led to a detailed understanding of the neural mechanisms that underlie decision making in primates. However, less is known about why such mechanisms are present in the first place. Theory suggests that primate decision making mechanisms, and their resultant behavioral abilities, emerged to maximize reward in the face of noisy, temporally evolving information. To test this theory, we trained an end-to-end deep recurrent neural network using reinforcement learning on a noisy perceptual discrimination task. Networks learned several key abilities of primate-like decision making including trading off speed for accuracy, and flexibly changing their mind in the face of new information. Internal dynamics of these networks suggest that these abilities were supported by similar decision mechanisms as those observed in primate neurophysiological studies. These results provide experimental support for key pressures that gave rise to the primate ability to make flexible decisions.