Learning to Allocate Limited Time to Decisions with Different Expected Outcomes

TL;DR

This study explores how humans allocate limited decision-making time based on expected outcomes, using behavioral experiments and computational models to understand adaptive threshold adjustments.

Contribution

It introduces a reinforcement learning model with time-varying thresholds that best explains human decision threshold adjustments under different reward conditions.

Findings

Participants often did not adopt optimal decision thresholds.

A reinforcement learning model with dynamic thresholds best fits the data.

Decision thresholds are adaptively adjusted based on feedback and trial difficulty.

Abstract

The goal of this article is to investigate how human participants allocate their limited time to decisions with different properties. We report the results of two behavioral experiments. In each trial of the experiments, the participant must accumulate noisy information to make a decision. The participants received positive and negative rewards for their correct and incorrect decisions, respectively. The stimulus was designed such that decisions based on more accumulated information were more accurate but took longer. Therefore, the total outcome that a participant could achieve during the limited experiments' time depended on her "decision threshold", the amount of information she needed to make a decision. In the first experiment, two types of trials were intermixed randomly: hard and easy. Crucially, the hard trials were associated with smaller positive and negative rewards than the easy trials. A cue presented at the beginning of each trial would indicate the type of the upcoming trial. The optimal strategy was to adopt a small decision threshold for hard trials. The results showed that several of the participants did not learn this simple strategy. We then investigated how the participants adjusted their decision threshold based on the feedback they received in each trial. To this end, we developed and compared 10 computational models for adjusting the decision threshold. The models differ in their assumptions on the shape of the decision thresholds and the way the feedback is used to adjust the decision thresholds. The results of Bayesian model comparison showed that a model with time-varying thresholds whose parameters are updated by a reinforcement learning algorithm is the most likely model.