Modelling Stock-market Investors as Reinforcement Learning Agents [Correction]

TL;DR

This study explores whether reinforcement learning models can explain stock-market investors' decision-making, finding that players tend to use a naive, short-sighted approach rather than sophisticated RL strategies.

Contribution

It demonstrates that a simple RL model can partially explain investor behavior, but overall investors appear to act naively, highlighting limitations of RL in modeling real-world trading decisions.

Findings

Simple RL models capture some investor behaviors

Players do not significantly improve decision-making with full RL models

Investors tend to act na"ively, focusing on immediate rewards

Abstract

Decision making in uncertain and risky environments is a prominent area of research. Standard economic theories fail to fully explain human behaviour, while a potentially promising alternative may lie in the direction of Reinforcement Learning (RL) theory. We analyse data for 46 players extracted from a financial market online game and test whether Reinforcement Learning (Q-Learning) could capture these players behaviour using a risk measure based on financial modeling. Moreover we test an earlier hypothesis that players are "na\"ive" (short-sighted). Our results indicate that a simple Reinforcement Learning model which considers only the selling component of the task captures the decision-making process for a subset of players but this is not sufficient to draw any conclusion on the population. We also find that there is not a significant improvement of fitting of the players when using a full RL model against a myopic version, where only immediate reward is valued by the players. This indicates that players, if using a Reinforcement Learning approach, do so na\"ively