Memory Encoding Model

TL;DR

This paper introduces a Memory Encoding Model that incorporates memory information to predict brain activity during vision-memory tasks, achieving top performance in a brain imaging competition and revealing periodic brain responses linked to memory processes.

Contribution

The paper presents a novel brain encoding model that integrates memory data, demonstrating superior prediction accuracy and uncovering periodic neural responses related to memory replay.

Findings

Memory encoding model outperformed existing models in brain prediction tasks.

Periodic delayed brain responses linked to previous images were observed.

Hippocampus activity correlated with periodic brain responses, suggesting a memory replay mechanism.

Abstract

We explore a new class of brain encoding model by adding memory-related information as input. Memory is an essential brain mechanism that works alongside visual stimuli. During a vision-memory cognitive task, we found the non-visual brain is largely predictable using previously seen images. Our Memory Encoding Model (Mem) won the Algonauts 2023 visual brain competition even without model ensemble (single model score 66.8, ensemble score 70.8). Our ensemble model without memory input (61.4) can also stand a 3rd place. Furthermore, we observe periodic delayed brain response correlated to 6th-7th prior image, and hippocampus also showed correlated activity timed with this periodicity. We conjuncture that the periodic replay could be related to memory mechanism to enhance the working memory.