Sequential sampling without comparison to boundary through model-free reinforcement learning

TL;DR

This paper introduces a model-free reinforcement learning approach for perceptual decision-making that eliminates the need for decision boundaries and evidence accumulation, explaining behavioral data without traditional boundary models.

Contribution

It presents a novel reinforcement learning algorithm that learns decision policies directly from evidence, bypassing the boundary concept and unifying learning and decision-making.

Findings

Reproduces key features of perceptual decision-making.

Accounts for behavior during training and after stabilization.

Offers a new perspective on decision process modeling.

Abstract

Although evidence integration to the boundary model has successfully explained a wide range of behavioral and neural data in decision making under uncertainty, how animals learn and optimize the boundary remains unresolved. Here, we propose a model-free reinforcement learning algorithm for perceptual decisions under uncertainty that dispenses entirely with the concepts of decision boundary and evidence accumulation. Our model learns whether to commit to a decision given the available evidence or continue sampling information at a cost. We reproduced the canonical features of perceptual decision-making such as dependence of accuracy and reaction time on evidence strength, modulation of speed-accuracy trade-off by payoff regime, and many others. By unifying learning and decision making within the same framework, this model can account for unstable behavior during training as well as stabilized post-training behavior, opening the door to revisiting the extensive volumes of discarded training data in the decision science literature.