Optimizing deep video representation to match brain activity

TL;DR

This paper investigates how deep learning models of video processing relate to brain activity, revealing complex visual area responses and a foveal/peripheral dichotomy through fMRI analysis.

Contribution

It introduces a method to compare deep video representations with brain activity, highlighting the neural correlates of visual processing.

Findings

Different visual areas show distinct responses to deep video features.

A notable foveal/peripheral dichotomy in visual cortex activity.

Deep representations correlate with specific brain activity patterns.

Abstract

The comparison of observed brain activity with the statistics generated by artificial intelligence systems is useful to probe brain functional organization under ecological conditions. Here we study fMRI activity in ten subjects watching color natural movies and compute deep representations of these movies with an architecture that relies on optical flow and image content. The association of activity in visual areas with the different layers of the deep architecture displays complexity-related contrasts across visual areas and reveals a striking foveal/peripheral dichotomy.