Does Learning Imply a Decrease in the Entropy of Behavior?

TL;DR

This paper investigates whether learning reduces the entropy of an organism's actions and perceptions, finding that initial entropy decreases but can later increase due to new options and changing states, questioning the straightforward link between learning and entropy reduction.

Contribution

The study provides empirical analysis of entropy changes during learning in a robot, highlighting complexities that challenge the assumption that learning always decreases entropy.

Findings

Entropy decreases initially during learning

Introduction of new options can increase entropy

Changing environmental states affect entropy dynamics

Abstract

Shannon's information entropy measures of the uncertainty of an event's outcome. If learning about a system reflects a decrease in uncertainty, then a plausible intuition is that learning should be accompanied by a decrease in the entropy of the organism's actions and/or perceptual states. To address whether this intuition is valid, I examined an artificial organism -- a simple robot -- that learned to navigate in an arena and analyzed the entropy of the outcome variables action, state, and reward. Entropy did indeed decrease in the initial stages of learning, but two factors complicated the scenario: (1) the introduction of new options discovered during the learning process and (2) the shifting patterns of perceptual and environmental states resulting from changes to the robot's learned movement strategies. These factors lead to a subsequent increase in entropy as the agent learned. I end with a discussion of the utility of information-based characterizations of learning.