Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems

TL;DR

This paper investigates human-machine teaming, revealing limitations of current AI methods and proposing interactive, explainable approaches that enhance collaboration, with empirical evidence guiding future research directions.

Contribution

It introduces mixed-initiative, interpretable AI teammates that improve collaboration, contrasting with brittle learning-based methods, and provides empirical guidelines for future HMT system development.

Findings

Learning-based methods underperform simple heuristics.

White-box, interactive approaches enhance team development.

Black-box models are easier to train and perform better.

Abstract

Collaborative robots and machine learning-based virtual agents are increasingly entering the human workspace with the aim of increasing productivity and enhancing safety. Despite this, we show in a ubiquitous experimental domain, Overcooked-AI, that state-of-the-art techniques for human-machine teaming (HMT), which rely on imitation or reinforcement learning, are brittle and result in a machine agent that aims to decouple the machine and human's actions to act independently rather than in a synergistic fashion. To remedy this deficiency, we develop HMT approaches that enable iterative, mixed-initiative team development allowing end-users to interactively reprogram interpretable AI teammates. Our 50-subject study provides several findings that we summarize into guidelines. While all approaches underperform a simple collaborative heuristic (a critical, negative result for learning-based methods), we find that white-box approaches supported by interactive modification can lead to significant team development, outperforming white-box approaches alone, and that black-box approaches are easier to train and result in better HMT performance highlighting a tradeoff between explainability and interactivity versus ease-of-training. Together, these findings present three important future research directions: 1) Improving the ability to generate collaborative agents with white-box models, 2) Better learning methods to facilitate collaboration rather than individualized coordination, and 3) Mixed-initiative interfaces that enable users, who may vary in ability, to improve collaboration.