Qualitative Event Perception: Leveraging Spatiotemporal Episodic Memory for Learning Combat in a Strategy Game

TL;DR

This paper presents a method for automatically segmenting continuous in-game experiences into meaningful episodes using spatiotemporal representations, which enhances a strategy game's agent learning and performance.

Contribution

It introduces a novel approach to event segmentation based on qualitative episodic memory, improving learning efficiency in a strategy game environment.

Findings

Episodes are segmented based on changing properties in the environment.

Spatiotemporal episode descriptions improve agent learning and gameplay performance.

Perception of spatial extent influences episode duration and case generation.

Abstract

Event perception refers to people's ability to carve up continuous experience into meaningful discrete events. We speak of finishing our morning coffee, mowing the lawn, leaving work, etc. as singular occurrences that are localized in time and space. In this work, we analyze how spatiotemporal representations can be used to automatically segment continuous experience into structured episodes, and how these descriptions can be used for analogical learning. These representations are based on Hayes' notion of histories and build upon existing work on qualitative episodic memory. Our agent automatically generates event descriptions of military battles in a strategy game and improves its gameplay by learning from this experience. Episodes are segmented based on changing properties in the world and we show evidence that they facilitate learning because they capture event descriptions at a useful spatiotemporal grain size. This is evaluated through our agent's performance in the game. We also show empirical evidence that the perception of spatial extent of episodes affects both their temporal duration as well as the number of overall cases generated.