MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data

TL;DR

MindAligner introduces an explicit brain functional alignment framework that enhances cross-subject visual decoding from limited fMRI data by learning a Brain Transfer Matrix and performing multi-level brain alignment.

Contribution

The paper proposes MindAligner, a novel framework that enables effective cross-subject brain decoding with limited data through explicit functional alignment and a Brain Transfer Matrix.

Findings

Outperforms existing methods in data-limited visual decoding

Provides high interpretability of brain functional correspondences

Enables use of pre-trained models across subjects

Abstract

Brain decoding aims to reconstruct visual perception of human subject from fMRI signals, which is crucial for understanding brain's perception mechanisms. Existing methods are confined to the single-subject paradigm due to substantial brain variability, which leads to weak generalization across individuals and incurs high training costs, exacerbated by limited availability of fMRI data. To address these challenges, we propose MindAligner, an explicit functional alignment framework for cross-subject brain decoding from limited fMRI data. The proposed MindAligner enjoys several merits. First, we learn a Brain Transfer Matrix (BTM) that projects the brain signals of an arbitrary new subject to one of the known subjects, enabling seamless use of pre-trained decoding models. Second, to facilitate reliable BTM learning, a Brain Functional Alignment module is proposed to perform soft cross-subject brain alignment under different visual stimuli with a multi-level brain alignment loss, uncovering fine-grained functional correspondences with high interpretability. Experiments indicate that MindAligner not only outperforms existing methods in visual decoding under data-limited conditions, but also provides valuable neuroscience insights in cross-subject functional analysis. The code will be made publicly available.