Free Will Belief as a consequence of Model-based Reinforcement Learning

TL;DR

This paper proposes that the human belief in free will arises from the way reinforcement learning models decision-making, where perceived freedom is linked to the entropy of action values and the need to model alternative choices for effective learning.

Contribution

It introduces a novel explanation for free will belief based on model-based reinforcement learning, connecting human cognition with computational decision-making frameworks.

Findings

Free will belief correlates with the entropy of RL agent action values.

Humans model themselves as if they could choose differently to solve the temporal credit assignment problem.

The RL framework explains why free will belief is widespread despite deterministic physical laws.

Abstract

The debate on whether or not humans have free will has been raging for centuries. Although there are good arguments based on our current understanding of the laws of nature for the view that it is not possible for humans to have free will, most people believe they do. This discrepancy begs for an explanation. If we accept that we do not have free will, we are faced with two problems: (1) while freedom is a very commonly used concept that everyone intuitively understands, what are we actually referring to when we say that an action or choice is "free" or not? And, (2) why is the belief in free will so common? Where does this belief come from, and what is its purpose, if any? In this paper, we examine these questions from the perspective of reinforcement learning (RL). RL is a framework originally developed for training artificial intelligence agents. However, it can also be used as a computational model of human decision making and learning, and by doing so, we propose that the first problem can be answered by observing that people's common sense understanding of freedom is closely related to the information entropy of an RL agent's normalized action values, while the second can be explained by the necessity for agents to model themselves as if they could have taken decisions other than those they actually took, when dealing with the temporal credit assignment problem. Put simply, we suggest that by applying the RL framework as a model for human learning it becomes evident that in order for us to learn efficiently and be intelligent we need to view ourselves as if we have free will.