Aligning brain functions boosts the decoding of visual semantics in novel subjects

TL;DR

This paper introduces a novel multi-subject alignment method for fMRI data that significantly improves the decoding of visual semantics across individuals, enabling better generalization and more efficient use of limited data.

Contribution

The study presents a new multi-subject alignment technique that enhances out-of-subject brain decoding performance and aligns neural representations with brain anatomy.

Findings

Out-of-subject decoding improved by up to 75% with alignment.

Our method outperforms classical approaches with less than 100 minutes of data.

Neural representations are aligned with brain anatomy.

Abstract

Deep learning is leading to major advances in the realm of brain decoding from functional Magnetic Resonance Imaging (fMRI). However, the large inter-subject variability in brain characteristics has limited most studies to train models on one subject at a time. Consequently, this approach hampers the training of deep learning models, which typically requires very large datasets. Here, we propose to boost brain decoding by aligning brain responses to videos and static images across subjects. Compared to the anatomically-aligned baseline, our method improves out-of-subject decoding performance by up to 75%. Moreover, it also outperforms classical single-subject approaches when fewer than 100 minutes of data is available for the tested subject. Furthermore, we propose a new multi-subject alignment method, which obtains comparable results to that of classical single-subject approaches while improving out-of-subject generalization. Finally, we show that this method aligns neural representations in accordance with brain anatomy. Overall, this study lays the foundations for leveraging extensive neuroimaging datasets and enhancing the decoding of individuals with a limited amount of brain recordings.