A Parameter-efficient Multi-subject Model for Predicting fMRI Activity

TL;DR

This paper introduces a parameter-efficient multi-subject model that predicts fMRI activity by combining shared and subject-specific linear transformations with a pretrained trunk, demonstrating effective encoding of neural data.

Contribution

The novel multi-subject linear encoding head efficiently captures shared and individual neural patterns using low-dimensional transformations and PCA embedding, improving fMRI prediction.

Findings

Model achieves accurate fMRI activity prediction across subjects.

Parameter efficiency reduces model complexity and training data requirements.

Open-source code facilitates reproducibility and further research.

Abstract

This is the Algonauts 2023 submission report for team "BlobGPT". Our model consists of a multi-subject linear encoding head attached to a pretrained trunk model. The multi-subject head consists of three components: (1) a shared multi-layer feature projection, (2) shared plus subject-specific low-dimension linear transformations, and (3) a shared PCA fMRI embedding. In this report, we explain these components in more detail and present some experimental results. Our code is available at https://github.com/cmi-dair/algonauts23.