Self-organized MT Direction Maps Emerge from Spatiotemporal Contrastive Optimization

TL;DR

This paper introduces a spatiotemporal deep learning model trained on natural videos that spontaneously develops brain-like direction maps and pinwheel structures, shedding light on cortical self-organization.

Contribution

It demonstrates that MT topography can emerge from a unified optimization principle balancing task performance and spatial regularization.

Findings

Model's direction maps match macaque MT physiological data

Emergence of pinwheel structures in the model

Direction selectivity arises from optimization trade-offs

Abstract

The spatial and functional organization of the primate visual cortex is a fundamental problem in neuroscience. While recent computational frameworks like the Topographic Deep Artificial Neural Network (TDANN) have successfully modeled spatial organization in the ventral stream, the computational origins of the dorsal stream's distinct topographies, such as direction-selective maps in the middle temporal (MT) area, remain largely unresolved. In this work, we present a spatiotemporal TDANN to investigate whether MT topography is governed by the same universal principles. By training a 3D ResNet on naturalistic videos via a Momentum Contrast (MoCo) self-supervised paradigm alongside a biologically inspired spatial loss, we demonstrate the spontaneous emergence of brain-like direction maps and topological pinwheel structures. Crucially, we reveal that MT tuning properties, characterized by strong direction selectivity paired with a residual axial component, arise from a strict optimization trade-off between task-driven discriminative pressure and spatial regularization. The model's representations quantitatively match in vivo macaque MT physiological baselines, including direction selectivity index, circular variance, and pinwheel density. These findings unify the computational origins of the ventral and dorsal streams, establishing a general mechanism for cortical self-organization.