Learning to Cooperate with Completely Unknown Teammates

TL;DR

This paper introduces a transfer learning approach combined with PLASTIC-Policy to enable agents to quickly adapt and cooperate with completely unknown teammates in ad hoc teamwork scenarios.

Contribution

It presents a novel method that effectively leverages transfer learning to improve cooperation with unseen teams, outperforming learning from scratch.

Findings

Transfer learning accelerates adaptation to new teammates.

The approach outperforms baseline methods in Half Field Offense.

Leveraging past knowledge improves cooperation efficiency.

Abstract

A key goal of ad hoc teamwork is to develop a learning agent that cooperates with unknown teams, without resorting to any pre-coordination protocol. Despite a vast number of ad hoc teamwork algorithms in the literature, most of them cannot address the problem of learning to cooperate with a completely unknown team, unless it learns from scratch. This article presents a novel approach that uses transfer learning alongside the state-of-the-art PLASTIC-Policy to adapt to completely unknown teammates quickly. We test our solution within the Half Field Offense simulator with five different teammates. The teammates were designed independently by developers from different countries and at different times. Our empirical evaluation shows that it is advantageous for an ad hoc agent to leverage its past knowledge when adapting to a new team instead of learning how to cooperate with it from scratch.