A Definition of Happiness for Reinforcement Learning Agents

TL;DR

This paper proposes a formal definition of happiness for reinforcement learning agents as the temporal difference error, aligning with human empirical research and satisfying key desiderata.

Contribution

It introduces a novel formal definition of happiness for RL agents based on temporal difference error, bridging AI and human happiness research.

Findings

The definition aligns with human empirical findings.

It satisfies most of the proposed desiderata.

Implications for AI and human happiness are discussed.

Abstract

What is happiness for reinforcement learning agents? We seek a formal definition satisfying a list of desiderata. Our proposed definition of happiness is the temporal difference error, i.e. the difference between the value of the obtained reward and observation and the agent's expectation of this value. This definition satisfies most of our desiderata and is compatible with empirical research on humans. We state several implications and discuss examples.