Making Friends in the Dark: Ad Hoc Teamwork Under Partial Observability

TL;DR

This paper formalizes ad hoc teamwork under partial observability and introduces a model-based approach that relies solely on prior knowledge and partial observations, enabling effective collaboration without access to teammate actions or reward signals.

Contribution

It presents the first formal definition and a novel model-based method for ad hoc teamwork under partial observability with specific assumptions, advancing the field.

Findings

Effective in assisting unknown teammates across 70 POMDPs

Robust scalability to more challenging problems

Outperforms previous approaches in partial observability settings

Abstract

This paper introduces a formal definition of the setting of ad hoc teamwork under partial observability and proposes a first-principled model-based approach which relies only on prior knowledge and partial observations of the environment in order to perform ad hoc teamwork. We make three distinct assumptions that set it apart previous works, namely: i) the state of the environment is always partially observable, ii) the actions of the teammates are always unavailable to the ad hoc agent and iii) the ad hoc agent has no access to a reward signal which could be used to learn the task from scratch. Our results in 70 POMDPs from 11 domains show that our approach is not only effective in assisting unknown teammates in solving unknown tasks but is also robust in scaling to more challenging problems.