Efficient Multi Subject Visual Reconstruction from fMRI Using Aligned Representations

TL;DR

This paper presents a new method for reconstructing visual images from fMRI data by aligning brain signals into a common, subject-agnostic space, improving efficiency and robustness especially with limited data.

Contribution

The work introduces a novel subject-agnostic common representation space for fMRI-based image reconstruction, enabling efficient alignment and transfer across subjects.

Findings

Effective in low-data scenarios

Subject and dataset-agnostic common space

Outperforms traditional end-to-end methods

Abstract

This work introduces a novel approach to fMRI-based visual image reconstruction using a subject-agnostic common representation space. We show that the brain signals of the subjects can be aligned in this common space during training to form a semantically aligned common brain. This is leveraged to demonstrate that aligning subject-specific lightweight modules to a reference subject is significantly more efficient than traditional end-to-end training methods. Our approach excels in low-data scenarios. We evaluate our methods on different datasets, demonstrating that the common space is subject and dataset-agnostic.