A Minimax Approach to Ad Hoc Teamwork

TL;DR

This paper introduces a minimax-Bayes approach for Ad Hoc Teamwork that optimizes policies against adversarial partner distributions, enhancing robustness in uncertain partner scenarios, demonstrated through experiments on cooking tasks.

Contribution

It presents a novel minimax-Bayes method that explicitly accounts for partner uncertainty, improving worst-case performance guarantees in ad hoc teamwork.

Findings

Outperforms self-play, fictitious play, and best response learning in robustness.

Shows effectiveness on Melting Pot cooking tasks.

Highlights importance of training distribution selection.

Abstract

We propose a minimax-Bayes approach to Ad Hoc Teamwork (AHT) that optimizes policies against an adversarial prior over partners, explicitly accounting for uncertainty about partners at time of deployment. Unlike existing methods that assume a specific distribution over partners, our approach improves worst-case performance guarantees. Extensive experiments, including evaluations on coordinated cooking tasks from the Melting Pot suite, show our method's superior robustness compared to self-play, fictitious play, and best response learning. Our work highlights the importance of selecting an appropriate training distribution over teammates to achieve robustness in AHT.