The order of multisensory associative sequences is reinstated as context feature during successful recognition

TL;DR

This study shows that the order in which sounds and images are experienced affects how they are remembered, with specific brain patterns linked to successful recognition.

Contribution

The study reveals a distinct neural signature tied to the sequence of multisensory associations during successful memory retrieval.

Findings

Oscillatory EEG activity showed a unique neural signature during successful recognition of modality sequences.

MVPA successfully decoded different modality sequence patterns, indicating specific memory traces.

The original sequence of sensory modalities influences memory retrieval and recognition.

Abstract

The ability of the human brain to encode and recognize sequential information from different sensory modalities is key to memory formation. The sequence in which these modalities are presented during encoding critically affects recognition. This study investigates the encoding of sensory modality sequences and its neural impact on recognition using multivariate pattern analysis (MVPA) of oscillatory EEG activity. We examined the reinstatement of multisensory episode-specific sequences in n = 32 participants who encoded sound-image associations (e.g., the image of a ship with the sound of a frog). Images and sounds were natural scenes and 2-second real-life sounds, presented sequentially during encoding. During recognition, stimulus pairs were presented simultaneously, and classification was used to test whether the modality sequence order could be decoded as a contextual feature in memory. Oscillatory results identified a distinct neural signature during successful retrieval, associated with the original modality sequence. Furthermore, MVPA successfully decoded neural patterns of different modality sequences, hinting at specific memory traces. These findings suggest that the sequence in which sensory modalities are encoded forms a neural signature, affecting later recognition. This study provides novel insights into the relationship between modality encoding and recognition, with broad implications for cognitive neuroscience and memory research.

Figures