Learning to Cooperate with Unseen Agent via Meta-Reinforcement Learning

TL;DR

This paper explores using meta-reinforcement learning to enable agents to learn cooperative skills from data, allowing them to adapt and cooperate effectively with unseen agents in complex environments.

Contribution

It introduces a meta-RL approach for ad hoc teamwork, demonstrating its effectiveness in enabling agents to learn cooperation without domain-specific knowledge.

Findings

Meta-RL produces robust cooperative agents in diverse environments.

Agents successfully adapt to different cooperative circumstances.

The method outperforms traditional approaches in ad hoc teamwork scenarios.

Abstract

Ad hoc teamwork problem describes situations where an agent has to cooperate with previously unseen agents to achieve a common goal. For an agent to be successful in these scenarios, it has to have a suitable cooperative skill. One could implement cooperative skills into an agent by using domain knowledge to design the agent's behavior. However, in complex domains, domain knowledge might not be available. Therefore, it is worthwhile to explore how to directly learn cooperative skills from data. In this work, we apply meta-reinforcement learning (meta-RL) formulation in the context of the ad hoc teamwork problem. Our empirical results show that such a method could produce robust cooperative agents in two cooperative environments with different cooperative circumstances: social compliance and language interpretation. (This is a full paper of the extended abstract version.)