Controlling Type Confounding in Ad Hoc Teamwork with Instance-wise Teammate Feedback Rectification

TL;DR

This paper introduces CTCAT, a causal inference-based method that rectifies instance-wise teammate feedback to control type confounding in ad hoc teamwork, improving agent robustness across various scenarios.

Contribution

It identifies the problem of type confounding in teammate modeling and proposes a novel rectification method to disentangle teammate instance distribution effects.

Findings

CTCAT effectively reduces the impact of type confounding.

The method improves robustness in classic and real-world ad hoc teamwork tasks.

Results demonstrate enhanced agent performance and stability.

Abstract

Ad hoc teamwork requires an agent to cooperate with unknown teammates without prior coordination. Many works propose to abstract teammate instances into high-level representation of types and then pre-train the best response for each type. However, most of them do not consider the distribution of teammate instances within a type. This could expose the agent to the hidden risk of \emph{type confounding}. In the worst case, the best response for an abstract teammate type could be the worst response for all specific instances of that type. This work addresses the issue from the lens of causal inference. We first theoretically demonstrate that this phenomenon is due to the spurious correlation brought by uncontrolled teammate distribution. Then, we propose our solution, CTCAT, which disentangles such correlation through an instance-wise teammate feedback rectification. This operation reweights the interaction of teammate instances within a shared type to reduce the influence of type confounding. The effect of CTCAT is evaluated in multiple domains, including classic ad hoc teamwork tasks and real-world scenarios. Results show that CTCAT is robust to the influence of type confounding, a practical issue that directly hazards the robustness of our trained agents but was unnoticed in previous works.