Model Cards for AI Teammates: Comparing Human-AI Team Familiarization Methods for High-Stakes Environments

TL;DR

This study compares three methods of familiarizing humans with AI teammates in high-stakes ISR environments, highlighting the trade-offs between speed of adoption, understanding, and risk behavior.

Contribution

It introduces a comparative analysis of familiarization methods and recommends a combined approach for effective human-AI team integration in critical settings.

Findings

Documentation speeds up strategy adoption but biases risk-averse behavior.

Direct interaction fosters risk-taking and experimentation, with less internal understanding.

Different user risk tolerances influence preferred AI interaction methods.

Abstract

We compare three methods of familiarizing a human with an artificial intelligence (AI) teammate ("agent") prior to operation in a collaborative, fast-paced intelligence, surveillance, and reconnaissance (ISR) environment. In a between-subjects user study (n=60), participants either read documentation about the agent, trained alongside the agent prior to the mission, or were given no familiarization. Results showed that the most valuable information about the agent included details of its decision-making algorithms and its relative strengths and weaknesses compared to the human. This information allowed the familiarization groups to form sophisticated team strategies more quickly than the control group. Documentation-based familiarization led to the fastest adoption of these strategies, but also biased participants towards risk-averse behavior that prevented high scores. Participants familiarized through direct interaction were able to infer much of the same information through observation, and were more willing to take risks and experiment with different control modes, but reported weaker understanding of the agent's internal processes. Significant differences were seen between individual participants' risk tolerance and methods of AI interaction, which should be considered when designing human-AI control interfaces. Based on our findings, we recommend a human-AI team familiarization method that combines AI documentation, structured in-situ training, and exploratory interaction.