RecBayes: Recurrent Bayesian Ad Hoc Teamwork in Large Partially Observable Domains

TL;DR

RecBayes introduces a recurrent Bayesian classifier for ad hoc teamwork in large, partially observable environments, enabling agents to identify teams and tasks solely from observations without access to environment states or teammates' actions.

Contribution

The paper presents RecBayes, a novel approach that handles large-scale, partially observable domains for ad hoc teamwork without requiring environment states or teammate actions, outperforming prior methods.

Findings

Effective in identifying teams and tasks from partial observations.

Scalable to environments with up to 1 million states.

Outperforms existing methods in large, partially observable benchmarks.

Abstract

This paper proposes RecBayes, a novel approach for ad hoc teamwork under partial observability, a setting where agents are deployed on-the-fly to environments where pre-existing teams operate, that never requires, at any stage, access to the states of the environment or the actions of its teammates. We show that by relying on a recurrent Bayesian classifier trained using past experiences, an ad hoc agent is effectively able to identify known teams and tasks being performed from observations alone. Unlike recent approaches such as PO-GPL (Gu et al., 2021) and FEAT (Rahman et al., 2023), that require at some stage fully observable states of the environment, actions of teammates, or both, or approaches such as ATPO (Ribeiro et al., 2023) that require the environments to be small enough to be tabularly modelled (Ribeiro et al., 2023), in their work up to 4.8K states and 1.7K observations, we show RecBayes is both able to handle arbitrarily large spaces while never relying on either states and teammates' actions. Our results in benchmark domains from the multi-agent systems literature, adapted for partial observability and scaled up to 1M states and 2^125 observations, show that RecBayes is effective at identifying known teams and tasks being performed from partial observations alone, and as a result, is able to assist the teams in solving the tasks effectively.