Unstable Dynamics of Adaptation in Unknown Environment due to Novelty Seeking

TL;DR

This paper explores how novelty-seeking behavior in a single reinforcement learning agent can lead to unstable adaptation dynamics and unexpected shifts from non-rational to rational behavior in unknown environments.

Contribution

It introduces a novel adaptation model incorporating novelty seeking, revealing potential instability and behavioral shifts not observed in simpler models.

Findings

Novelty seeking can cause inherent instability in single-agent adaptation.

Increased novelty-seeking may trigger a switch from non-rational to rational behavior.

Intrinsic motives significantly influence complex socio-economic dynamics.

Abstract

Learning and adaptation play great role in emergent socio-economic phenomena. Complex dynamics has been previously found in the systems of multiple learning agents interacting via a simple game. Meanwhile, the single agent adaptation is considered trivially stable. We advocate the idea that adopting a more complex model of the individual behavior may result in a more diverse spectrum of macro-level behaviors. We develop an adaptation model based on the reinforcement learning framework extended by an additional processing channel. We scrutiny the dynamics of the single agent adapting to the unknown environment; the agent is biased by novelty seeking, the intrinsic inclination for exploration. We demonstrate that the behavior of the novelty-seeking agent may be inherently unstable. One of the surprising results is that under certain conditions the increase of the novelty-seeking level may cause the agent to switch from the non-rational to the strictly rational behavior. Our results give evidence to the hypothesis that the intrinsic motives of agents should be paid no less attention than the extrinsic ones in the models of complex socio-economic systems.