Balancing New Against Old Information: The Role of Surprise in Learning

TL;DR

This paper introduces a surprise measure for learning that balances new and old information, enabling adaptive learning in changing environments without prior knowledge of environmental dynamics.

Contribution

It proposes a novel surprise measure based on data likelihood and belief entropy, and demonstrates its effectiveness in dynamic decision-making and maze exploration tasks.

Findings

Surprise-minimizing learning adapts to environmental changes.

Framework performs well in complex, changing environments.

Potential applications in modeling human and animal responses to surprise.

Abstract

Surprise describes a range of phenomena from unexpected events to behavioral responses. We propose a measure of surprise and use it for surprise-driven learning. Our surprise measure takes into account data likelihood as well as the degree of commitment to a belief via the entropy of the belief distribution. We find that surprise-minimizing learning dynamically adjusts the balance between new and old information without the need of knowledge about the temporal statistics of the environment. We apply our framework to a dynamic decision-making task and a maze exploration task. Our surprise minimizing framework is suitable for learning in complex environments, even if the environment undergoes gradual or sudden changes and could eventually provide a framework to study the behavior of humans and animals encountering surprising events.