Decision-making with distorted memory: Escaping the trap of past experience

TL;DR

This paper models how decision-makers influenced by peak experiences can become trapped in suboptimal choices, revealing conditions under which noise optimizes long-term outcomes using statistical physics tools.

Contribution

It extends previous models to include different utility distributions and analyzes how noise levels affect long-term decision outcomes.

Findings

Agents can become trapped in choices based on early experiences.

Optimal noise level exists that maximizes expected long-term returns.

Different utility distributions influence decision dynamics and outcomes.

Abstract

Snapshots of "best" (or "worst") experience are known to dominate human memory and may thus also have a significant effect on future behaviour. We consider here a model of repeated decision-making where, at every time step, an agent takes one of two choices with probabilities which are functions of the maximum utilities previously experienced. Depending on the utility distributions and the level of noise in the decision process, it is possible for an agent to become "trapped" in one of the choices on the basis of their early experiences. If the utility distributions for the two choices are different, then the agent may even become trapped in the choice which is objectively worse in the sense of expected long-term returns; crucially we extend earlier work to address this case. Using tools from statistical physics and extreme-value theory, we show that for exponential utilities there is an optimal value of noise which maximizes the expected returns in the long run. We also briefly discuss the behaviour for other utility distributions.