# Estimation-uncertainty affects decisions with and without learning opportunities

## TL;DR

The paper shows that how often an option is sampled during learning affects later decisions, even without feedback or exploration.

## Contribution

The study reveals that estimation-uncertainty influences long-term decisions independently of outcome expectations.

## Key findings

- Sampling rates during learning correlate with decision biases in a later test phase without feedback.
- Estimation-uncertainty improves computational model fits, especially for less-sampled options.
- Results are replicated across three independent datasets.

## Abstract

Motivated behavior during reinforcement learning is determined by outcome expectations and their estimation-uncertainty (how frequently an option has been sampled), with the latter modulating exploration rates. However, although differences in sampling-rates are inherent to most types of reinforcement learning paradigms that confront highly rewarded options with less rewarded ones, it is unclear whether and how estimation-uncertainty lingers to affect long-term decisions without opportunities to learn or to explore. Here, we show that sampling-rates acquired during a reinforcement learning phase (with feedback) correlate with decision biases in a subsequent test phase (without feedback), independently from outcome expectations. Further, computational model-fits to behavior are improved by estimation-uncertainty, and specifically so for options with smaller sampling-rates/larger estimation-uncertainties. These results are replicated in two additional independent datasets. Our findings highlight that estimation-uncertainty is an important factor to consider when trying to understand human decision making.

Decisions are often assumed to depend on expected outcomes alone, with more profitable actions being favored. Here, the authors show that outcome uncertainty also shapes choices, such that less-sampled actions are avoided, independently of their value.