CNeuroMod-THINGS, a densely-sampled fMRI dataset for visual neuroscience

TL;DR

CNeuroMod-THINGS is a large-scale, densely-sampled fMRI dataset combining well-annotated images and neural responses to advance visual neuroscience and neuro-AI modeling.

Contribution

It integrates existing image and neural response datasets to create a comprehensive resource for studying human visual representations.

Findings

High-quality neuroimaging and behavioral data collected.

Dataset covers 720 categories with 4000 images.

Enables improved modeling of visual neural responses.

Abstract

Data-hungry neuro-AI modelling requires ever larger neuroimaging datasets. CNeuroMod-THINGS meets this need by capturing neural representations for a wide set of semantic concepts using well-characterized images in a new densely-sampled, large-scale fMRI dataset. Importantly, CNeuroMod-THINGS exploits synergies between two existing projects: the THINGS initiative (THINGS) and the Courtois Project on Neural Modelling (CNeuroMod). THINGS has developed a common set of thoroughly annotated images broadly sampling natural and man-made objects which is used to acquire a growing collection of large-scale multimodal neural responses. Meanwhile, CNeuroMod is acquiring hundreds of hours of fMRI data from a core set of participants during controlled and naturalistic tasks, including visual tasks like movie watching and videogame playing. For CNeuroMod-THINGS, four CNeuroMod participants each completed 33-36 sessions of a continuous recognition paradigm using approximately 4000 images from the THINGS stimulus set spanning 720 categories. We report behavioural and neuroimaging metrics that showcase the quality of the data. By bridging together large existing resources, CNeuroMod-THINGS expands our capacity to model broad slices of the human visual experience.