Strategy Extraction in Single-Agent Games

TL;DR

This paper introduces a method for extracting transferable strategies from single-agent game environments by analyzing event sequences, aiming to enhance generalization and transfer learning in AI agents.

Contribution

It presents a novel approach combining event frequency analysis with sequence alignment to identify strategies that can be applied across different contexts.

Findings

Successfully identified plausible strategies in three different games.

Demonstrated potential for strategy extraction to support transfer learning.

Provides a foundation for future research on generalization in AI agents.

Abstract

The ability to continuously learn and adapt to new situations is one where humans are far superior compared to AI agents. We propose an approach to knowledge transfer using behavioural strategies as a form of transferable knowledge influenced by the human cognitive ability to develop strategies. A strategy is defined as a partial sequence of events - where an event is both the result of an agent's action and changes in state - to reach some predefined event of interest. This information acts as guidance or a partial solution that an agent can generalise and use to make predictions about how to handle unknown observed phenomena. As a first step toward this goal, we develop a method for extracting strategies from an agent's existing knowledge that can be applied in multiple contexts. Our method combines observed event frequency information with local sequence alignment techniques to find patterns of significance that form a strategy. We show that our method can identify plausible strategies in three environments: Pacman, Bank Heist and a dungeon-crawling video game. Our evaluation serves as a promising first step toward extracting knowledge for generalisation and, ultimately, transfer learning.