Enhancing Joint Human-AI Inference in Robot Missions: A Confidence-Based Approach

TL;DR

This paper introduces a confidence-based method for joint human-AI inference in robot missions, demonstrating improved accuracy and the importance of well-calibrated AI confidence for effective human-AI collaboration.

Contribution

It presents the first application of a maximum-confidence heuristic for joint inference in robot teleoperation, highlighting the impact of AI confidence calibration on performance.

Findings

Joint inference accuracy improves with better AI confidence calibration.

Humans adapt their inferences based on AI recommendations and confidence levels.

Poorly calibrated AI systems can harm team performance.

Abstract

Joint human-AI inference holds immense potential to improve outcomes in human-supervised robot missions. Current day missions are generally in the AI-assisted setting, where the human operator makes the final inference based on the AI recommendation. However, due to failures in human judgement on when to accept or reject the AI recommendation, complementarity is rarely achieved. We investigate joint human-AI inference where the inference made with higher confidence is selected. Through a user study with N=100 participants on a representative simulated robot teleoperation task, specifically studying the inference of robots' control delays we show that: a) Joint inference accuracy is higher and its extent is regulated by the confidence calibration of the AI agent, and b) Humans change their inferences based on AI recommendations and the extent and direction of this change is also regulated by the confidence calibration of the AI agent. Interestingly, our results show that pairing poorly-calibrated AI-DSS with humans hurts performance instead of helping the team, reiterating the need for AI-based decision support systems with good metacognitive sensitivity. To the best of our knowledge, our study presents the first application of a maximum-confidence-based heuristic for joint human-AI inference within a simulated robot teleoperation task.