CM-DQN: A Value-Based Deep Reinforcement Learning Model to Simulate Confirmation Bias

TL;DR

This paper introduces CM-DQN, a deep reinforcement learning model that simulates human confirmation bias by applying different update strategies for positive and negative prediction errors, demonstrating improved learning effects in experiments.

Contribution

The paper proposes a novel deep reinforcement learning algorithm, CM-DQN, that models confirmation bias by differentially updating based on outcome valence, advancing understanding of human-like decision-making.

Findings

Confirmatory bias improves learning performance.

CM-DQN effectively simulates human decision-making biases.

Experimental results show enhanced learning with confirmation bias.

Abstract

In human decision-making tasks, individuals learn through trials and prediction errors. When individuals learn the task, some are more influenced by good outcomes, while others weigh bad outcomes more heavily. Such confirmation bias can lead to different learning effects. In this study, we propose a new algorithm in Deep Reinforcement Learning, CM-DQN, which applies the idea of different update strategies for positive or negative prediction errors, to simulate the human decision-making process when the task's states are continuous while the actions are discrete. We test in Lunar Lander environment with confirmatory, disconfirmatory bias and non-biased to observe the learning effects. Moreover, we apply the confirmation model in a multi-armed bandit problem (environment in discrete states and discrete actions), which utilizes the same idea as our proposed algorithm, as a contrast experiment to algorithmically simulate the impact of different confirmation bias in decision-making process. In both experiments, confirmatory bias indicates a better learning effect.