Perception Activator: An intuitive and portable framework for brain cognitive exploration
Le Xu, Qi Zhang, Qixian Zhang, Hongyun Zhang, Duoqian Miao, Cairong Zhao

TL;DR
This paper introduces Perception Activator, a portable framework that enhances brain-vision decoding by integrating fMRI signals into multi-scale image features, improving semantic object reconstruction and understanding of visual perception.
Contribution
The paper presents a novel framework that uses fMRI data to improve semantic alignment in brain-vision decoding, addressing limitations of pixel-level approaches.
Findings
Incorporating fMRI signals improves object detection accuracy.
fMRI contains rich multi-object semantic cues.
The framework enhances semantic object reconstruction fidelity.
Abstract
Recent advances in brain-vision decoding have driven significant progress, reconstructing with high fidelity perceived visual stimuli from neural activity, e.g., functional magnetic resonance imaging (fMRI), in the human visual cortex. Most existing methods decode the brain signal using a two-level strategy, i.e., pixel-level and semantic-level. However, these methods rely heavily on low-level pixel alignment yet lack sufficient and fine-grained semantic alignment, resulting in obvious reconstruction distortions of multiple semantic objects. To better understand the brain's visual perception patterns and how current decoding models process semantic objects, we have developed an experimental framework that uses fMRI representations as intervention conditions. By injecting these representations into multi-scale image features via cross-attention, we compare both downstream performance and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
