# A 7T fMRI dataset of synthetic images for out-of-distribution modeling of vision

## TL;DR

Researchers created a new dataset of synthetic images and fMRI responses to test and improve the robustness of brain models under out-of-distribution conditions.

## Contribution

They released NSD-synthetic, a dataset of 7T fMRI responses for synthetic images to enable out-of-distribution generalization testing.

## Key findings

- NSD-synthetic fMRI responses reliably encode stimulus-related information and are out-of-distribution relative to NSD.
- OOD generalization tests on NSD-synthetic reveal differences between brain models not detected in-distribution.
- The degree of OOD is predictive of the magnitude of model failures.

## Abstract

Large-scale visual neural datasets such as the Natural Scenes Dataset (NSD) are enabling models of the brain with performances beyond what was possible just a decade ago. However, because the stimuli of these datasets typically live within a common naturalistic visual distribution, they make it challenging to implement out-of-distribution (OOD) generalization tests crucial for the development of robust brain models. Here, we address this by releasing NSD-synthetic, a dataset of 7T fMRI responses from the same eight NSD participants for 284 synthetic images. We show that NSD-synthetic’s fMRI responses reliably encode stimulus-related information and are OOD with respect to NSD; that OOD generalization tests on NSD-synthetic reveal differences between brain models that are not detected in-distribution; and that the degree of OOD (quantified as the test data distance from the training data) is predictive of the magnitude of model failures. Together, NSD-synthetic enables OOD generalization tests that facilitate the development of more robust models of visual processing.

The authors released NSD-synthetic, a dataset of 7T fMRI responses from the same eight NSD participants for 284 out-of-distribution synthetic images, to facilitate the development of more robust models of visual processing.