Comparing Psychometric and Behavioral Predictors of Compliance During Human-AI Interactions

TL;DR

This study compares self-reported trust measures with behavioral predictors to forecast human compliance in human-AI interactions, finding behavioral data more effective and suggesting practical advantages for AI adaptation.

Contribution

It demonstrates that behavioral predictors outperform traditional trust inventories in predicting compliance, highlighting a new approach for adaptive AI systems.

Findings

Behavioral measures predict compliance better than trust inventories.

Self-report inventories are less effective than behavioral data.

Behavioral data can be a cost-effective alternative for AI adaptation.

Abstract

Optimization of human-AI teams hinges on the AI's ability to tailor its interaction to individual human teammates. A common hypothesis in adaptive AI research is that minor differences in people's predisposition to trust can significantly impact their likelihood of complying with recommendations from the AI. Predisposition to trust is often measured with self-report inventories that are administered before interactions. We benchmark a popular measure of this kind against behavioral predictors of compliance. We find that the inventory is a less effective predictor of compliance than the behavioral measures in datasets taken from three previous research projects. This suggests a general property that individual differences in initial behavior are more predictive than differences in self-reported trust attitudes. This result also shows a potential for easily accessible behavioral measures to provide an AI with more accurate models without the use of (often costly) survey instruments.