Auditory Resource Redistribution in Audiovisual Integration: Evidence from Attribute Amnesia

TL;DR

This study explores how combining auditory and visual information improves visual recognition but may lead to forgetting some auditory details.

Contribution

The study provides evidence that audiovisual integration enhances visual recognition at the cost of auditory information loss via attribute amnesia.

Findings

Both pitch and semantic auditory attributes were subject to attribute amnesia, with semantic cues recalled more accurately.

Semantic congruency improved target identification, showing automatic processing of semantic auditory cues.

Improved recall of auditory attributes in post-surprise trials supports the working memory reselection model.

Abstract

Auditory stimuli are known to enhance visual target recognition in rapid serial visual presentation (RSVP) tasks, yet the robustness and potential trade-offs of this audiovisual integration (AVI) effect remain debated. Attribute amnesia (AA) refers to the phenomenon in which individuals successfully identify a stimulus for a task, but fail to recall its basic attributes when unexpectedly tested. The present study investigates whether improvements in visual recognition through AVI occur at the expense of auditory information loss, as predicted by the AA framework. Across two RSVP experiments, participants were presented with letter targets embedded among digit distractors. In Experiment 1, an auditory pitch (bass, alto, treble) accompanied the second target (T2); in Experiment 2, an auditory syllable either matched or mismatched the semantic identity of T2. A surprise-test paradigm was used to assess participants’ ability to recall auditory stimuli. The results show that both pitch and semantic attributes were subject to AA, with semantic stimuli recalled more accurately than pitch. Moreover, semantic congruency enhanced T2 identification, highlighting the automatic processing advantage of semantic cues. Post-surprise trials revealed the improved recall of auditory attributes, consistent with the working memory reselection model. Together, these findings suggest that AVI enhances visual recognition by reallocating cognitive resources, but at the cost of a partial loss of irrelevant auditory information.

Figures