On the Utility of External Agent Intention Predictor for Human-AI Coordination

TL;DR

This paper introduces a two-stage paradigm using a Theory of Mind model to predict AI intentions, enhancing human-AI coordination and situational awareness through real-time action predictions.

Contribution

It proposes a novel two-stage approach with a transformer-based predictor to improve human understanding of AI intentions without modifying the AI itself.

Findings

Improved team performance with intention prediction

Enhanced human situational awareness

Versatile method applicable to various AI agents

Abstract

Reaching a consensus on the team plans is vital to human-AI coordination. Although previous studies provide approaches through communications in various ways, it could still be hard to coordinate when the AI has no explainable plan to communicate. To cover this gap, we suggest incorporating external models to assist humans in understanding the intentions of AI agents. In this paper, we propose a two-stage paradigm that first trains a Theory of Mind (ToM) model from collected offline trajectories of the target agent, and utilizes the model in the process of human-AI collaboration by real-timely displaying the future action predictions of the target agent. Such a paradigm leaves the AI agent as a black box and thus is available for improving any agents. To test our paradigm, we further implement a transformer-based predictor as the ToM model and develop an extended online human-AI collaboration platform for experiments. The comprehensive experimental results verify that human-AI teams can achieve better performance with the help of our model. A user assessment attached to the experiment further demonstrates that our paradigm can significantly enhance the situational awareness of humans. Our study presents the potential to augment the ability of humans via external assistance in human-AI collaboration, which may further inspire future research.