Improving the State of the Art for Training Human-AI Teams: Technical Report #5 -- Individual Differences and Team Qualities to Measure in a Human-AI Teaming Testbed

TL;DR

This technical report discusses developing a testbed for human-AI teaming, focusing on measuring individual differences and team qualities through surveys and constructs to enhance training and evaluation.

Contribution

It introduces a framework for assessing individual differences and team qualities in a human-AI teaming environment within a Synthetic Task Environment.

Findings

Reviewed key constructs for individual differences and team qualities

Explored measurement methods within the STE

Supports data collection via surveys before and after performance

Abstract

Sonalysts, Inc. (Sonalysts) is working on an initiative to expand our expertise in teaming to include Human-Artificial Intelligence (AI) teams. The first step of this process is to develop a Synthetic Task Environment (STE) to support our original research. Prior knowledge elicitation efforts within the Human-AI teaming research stakeholder community revealed a desire to support data collection using pre- and post-performance surveys. In this technical report, we review a number of constructs that capture meaningful individual differences and teaming qualities. Additionally, we explore methods of measuring those constructs within the STE.