The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes

TL;DR

The paper introduces the 2023 Algonauts Project challenge, aiming to develop computational models of the visual brain using a large fMRI dataset of natural scenes, fostering collaboration between biological and artificial intelligence research.

Contribution

It presents a new challenge framework utilizing the Natural Scenes Dataset to promote data-driven modeling of the visual cortex, encouraging interdisciplinary collaboration.

Findings

Public leaderboard for model comparison

Rich dataset of 73,000 natural scene fMRI responses

Facilitates rapid development of brain-inspired AI models

Abstract

The sciences of biological and artificial intelligence are ever more intertwined. Neural computational principles inspire new intelligent machines, which are in turn used to advance theoretical understanding of the brain. To promote further exchange of ideas and collaboration between biological and artificial intelligence researchers, we introduce the 2023 installment of the Algonauts Project challenge: How the Human Brain Makes Sense of Natural Scenes (http://algonauts.csail.mit.edu). This installment prompts the fields of artificial and biological intelligence to come together towards building computational models of the visual brain using the largest and richest dataset of fMRI responses to visual scenes, the Natural Scenes Dataset (NSD). NSD provides high-quality fMRI responses to ~73,000 different naturalistic colored scenes, making it the ideal candidate for data-driven model building approaches promoted by the 2023 challenge. The challenge is open to all and makes results directly comparable and transparent through a public leaderboard automatically updated after each submission, thus allowing for rapid model development. We believe that the 2023 installment will spark symbiotic collaborations between biological and artificial intelligence scientists, leading to a deeper understanding of the brain through cutting-edge computational models and to novel ways of engineering artificial intelligent agents through inductive biases from biological systems.