Learning to Act and Observe in Partially Observable Domains

TL;DR

This paper introduces learning algorithms for agents in partially observable environments, enabling them to learn observable aspects and action effects using dynamic epistemic logic, extending previous work on fully observable domains.

Contribution

It presents novel DEL-based algorithms for learning in partially observable settings, differentiating levels of domain knowledge and observational requirements.

Findings

Algorithms effectively learn observable domain features.

Agents understand action effects in partial observability.

Extension of DEL-based learning to new domain types.

Abstract

We consider a learning agent in a partially observable environment, with which the agent has never interacted before, and about which it learns both what it can observe and how its actions affect the environment. The agent can learn about this domain from experience gathered by taking actions in the domain and observing their results. We present learning algorithms capable of learning as much as possible (in a well-defined sense) both about what is directly observable and about what actions do in the domain, given the learner's observational constraints. We differentiate the level of domain knowledge attained by each algorithm, and characterize the type of observations required to reach it. The algorithms use dynamic epistemic logic (DEL) to represent the learned domain information symbolically. Our work continues that of Bolander and Gierasimczuk (2015), which developed DEL-based learning algorithms based to learn domain information in fully observable domains.