Human-Agent Coordination in Games under Incomplete Information via Multi-Step Intent

TL;DR

This paper introduces a multi-step intent approach with an online planning algorithm, IntentMCTS, to improve human-agent coordination in incomplete information games, leading to higher success rates and lower user frustration.

Contribution

It extends shared-control games to multi-step actions and develops IntentMCTS, a novel online planning method utilizing multi-step intent and belief modeling for better cooperation.

Findings

IntentMCTS reduces steps and control switches in simulations.

Human-agent study shows 18.52% higher success rate with IntentMCTS.

Participants report lower cognitive load and frustration.

Abstract

Strategic coordination between autonomous agents and human partners under incomplete information can be modeled as turn-based cooperative games. We extend a turn-based game under incomplete information, the shared-control game, to allow players to take multiple actions per turn rather than a single action. The extension enables the use of multi-step intent, which we hypothesize will improve performance in long-horizon tasks. To synthesize cooperative policies for the agent in this extended game, we propose an approach featuring a memory module for a running probabilistic belief of the environment dynamics and an online planning algorithm called IntentMCTS. This algorithm strategically selects the next action by leveraging any communicated multi-step intent via reward augmentation while considering the current belief. Agent-to-agent simulations in the Gnomes at Night testbed demonstrate that IntentMCTS requires fewer steps and control switches than baseline methods. A human-agent user study corroborates these findings, showing an 18.52% higher success rate compared to the heuristic baseline and a 5.56% improvement over the single-step prior work. Participants also report lower cognitive load, frustration, and higher satisfaction with the IntentMCTS agent partner.