Tracking Emotions: Intrinsic Motivation Grounded on Multi-Level Prediction Error Dynamics

TL;DR

This paper introduces an intrinsic motivation architecture for artificial agents that uses multi-level prediction error dynamics to regulate goal selection, exploration, and exploitation, leading to improved learning performance.

Contribution

It proposes a novel architecture that dynamically monitors prediction error to motivate goal-directed behavior and balance exploration and exploitation in artificial agents.

Findings

Outperforms fixed-noise intrinsic motivation methods.

Effectively balances exploration and exploitation.

Enhances goal adaptation through prediction error monitoring.

Abstract

How do cognitive agents decide what is the relevant information to learn and how goals are selected to gain this knowledge? Cognitive agents need to be motivated to perform any action. We discuss that emotions arise when differences between expected and actual rates of progress towards a goal are experienced. Therefore, the tracking of prediction error dynamics has a tight relationship with emotions. Here, we suggest that the tracking of prediction error dynamics allows an artificial agent to be intrinsically motivated to seek new experiences but constrained to those that generate reducible prediction error.We present an intrinsic motivation architecture that generates behaviors towards self-generated and dynamic goals and that regulates goal selection and the balance between exploitation and exploration through multi-level monitoring of prediction error dynamics. This new architecture modulates exploration noise and leverages computational resources according to the dynamics of the overall performance of the learning system. Additionally, it establishes a possible solution to the temporal dynamics of goal selection. The results of the experiments presented here suggest that this architecture outperforms intrinsic motivation approaches where exploratory noise and goals are fixed and a greedy strategy is applied.