Team Behavior in Interactive Dynamic Influence Diagrams with Applications to Ad Hoc Teams

TL;DR

This paper explores how integrating learning into planning within interactive dynamic influence diagrams enhances ad hoc team cooperation, addressing the limitations of finitely-nested models in multiagent decision-making.

Contribution

It introduces a method that incorporates learning at level 0 models to improve team optimality in ad hoc teamwork scenarios.

Findings

Learning integrated into planning facilitates optimal team behavior.

Finitely-nested models may not achieve optimal cooperation.

The approach is applicable to ad hoc teamwork settings.

Abstract

Planning for ad hoc teamwork is challenging because it involves agents collaborating without any prior coordination or communication. The focus is on principled methods for a single agent to cooperate with others. This motivates investigating the ad hoc teamwork problem in the context of individual decision making frameworks. However, individual decision making in multiagent settings faces the task of having to reason about other agents' actions, which in turn involves reasoning about others. An established approximation that operationalizes this approach is to bound the infinite nesting from below by introducing level 0 models. We show that a consequence of the finitely-nested modeling is that we may not obtain optimal team solutions in cooperative settings. We address this limitation by including models at level 0 whose solutions involve learning. We demonstrate that the learning integrated into planning in the context of interactive dynamic influence diagrams facilitates optimal team behavior, and is applicable to ad hoc teamwork.