Fast, Accurate, but Sometimes Too-Compelling Support: The Impact of Imperfectly Automated Cues in an Augmented Reality Head-Mounted Display on Visual Search Performance

TL;DR

This study evaluates how different augmented reality cues influence visual search performance and automation bias, revealing benefits of location-based cues but also highlighting risks of over-reliance on imperfect AR automation.

Contribution

It compares the effectiveness of arrow, minimap, and icon cues in AR headsets and examines the impact of automation errors on search performance and bias.

Findings

Location cues improved search speed and accuracy over icon cues.

Automation bias increased with more reliable cues, leading to blind trust.

Incorrect cues caused performance declines, especially with location-based cues.

Abstract

While visual search for targets within a complex scene might benefit from using augmented-reality (AR) head-mounted display (HMD) technologies helping to efficiently direct human attention, imperfectly reliable automation support could manifest in occasional errors. The current study examined the effectiveness of different HMD cues that might support visual search performance and their respective consequences following automation errors. Fifty-six participants searched a 3D environment containing 48 objects in a room, in order to locate a target object that was viewed prior to each trial. They searched either unaided or assisted by one of three HMD types of cues: an arrow pointing to the target, a plan-view minimap highlighting the target, and a constantly visible icon depicting the appearance of the target object. The cue was incorrect on 17% of the trials for one group of participants and 100% correct for the second group. Through both analysis and modeling of both search speed and accuracy, the results indicated that the arrow and minimap cues depicting location information were more effective than the icon cue depicting visual appearance, both overall, and when the cue was correct. However, there was a tradeoff on the infrequent occasions when the cue erred. The most effective AR-based cue led to a greater automation bias, in which the cue was more often blindly followed without careful examination of the raw images. The results speak to the benefits of augmented reality and the need to examine potential costs when AR-conveyed information may be incorrect because of imperfectly reliable systems.