Do oral messages help visual search?

TL;DR

This study investigates how oral messages, especially multimodal ones combining visual and verbal cues, can enhance the speed and accuracy of visual search tasks on crowded displays.

Contribution

It demonstrates that well-designed verbal messages, particularly multimodal cues, significantly improve visual search performance and are highly rated by users.

Findings

Multimodal messages improve search speed and accuracy.

Verbal cues are rated highly by users.

Visual presentation combined with oral information is most effective.

Abstract

A preliminary experimental study is presented, that aims at eliciting the contribution of oral messages to facilitating visual search tasks on crowded visual displays. Results of quantitative and qualitative analyses suggest that appropriate verbal messages can improve both target selection time and accuracy. In particular, multimodal messages including a visual presentation of the isolated target together with absolute spatial oral information on its location in the displayed scene seem most effective. These messages also got top-ranking ratings from most subjects.