Modeling Spatiotemporal Neural Frames for High Resolution Brain Dynamic

TL;DR

This paper introduces an EEG-conditioned framework for high-resolution fMRI reconstruction, leveraging multimodal data to improve spatial and temporal fidelity of brain activity sequences.

Contribution

It presents a novel method combining EEG and fMRI data with null-space reconstruction to enhance dynamic brain activity modeling.

Findings

Superior voxel-wise reconstruction quality demonstrated on CineBrain dataset.

Robust temporal consistency across whole-brain and specific regions achieved.

Reconstructed fMRI preserves functional information for downstream decoding.

Abstract

Capturing dynamic spatiotemporal neural activity is essential for understanding large-scale brain mechanisms. Functional magnetic resonance imaging (fMRI) provides high-resolution cortical representations that form a strong basis for characterizing fine-grained brain activity patterns. The high acquisition cost of fMRI limits large-scale applications, therefore making high-quality fMRI reconstruction a crucial task. Electroencephalography (EEG) offers millisecond-level temporal cues that complement fMRI. Leveraging this complementarity, we present an EEG-conditioned framework for reconstructing dynamic fMRI as continuous neural sequences with high spatial fidelity and strong temporal coherence at the cortical-vertex level. To address sampling irregularities common in real fMRI acquisitions, we incorporate a null-space intermediate-frame reconstruction, enabling measurement-consistent completion of arbitrary intermediate frames and improving sequence continuity and practical applicability. Experiments on the CineBrain dataset demonstrate superior voxel-wise reconstruction quality and robust temporal consistency across whole-brain and functionally specific regions. The reconstructed fMRI also preserves essential functional information, supporting downstream visual decoding tasks. This work provides a new pathway for estimating high-resolution fMRI dynamics from EEG and advances multimodal neuroimaging toward more dynamic brain activity modeling.