Integrative characterization of the topography of V4 neural codes using deep learning approaches

TL;DR

This study uses deep learning models fitted to large-scale neural imaging data to systematically explore how V4 neurons encode various visual features and how these preferences are organized topographically.

Contribution

It introduces a novel approach combining deep neural network models with wide-field calcium imaging to map V4 neural codes across multiple visual attributes.

Findings

Neurons preferring 2D shapes also prefer 3D surface features.

Texture-preferring neurons favor flat, uniform surfaces.

V4 encodes shape and surface appearance in interleaved clusters.

Abstract

Area V4 is a mid-level stage of the macaque ventral visual stream, known to encode intermediate visual features such as color, curvature, corners, texture, three-dimensional (3D) solids, and local form. Classical neurophysiological studies have typically examined these dimensions in isolation, contrasting V4 selectivity for shape versus texture, 3D solid surfaces versus two-dimensional (2D) flat patterns, or object form versus texture. Yet how these tunings relate to one another within individual neurons, and how they are jointly organized across the cortical surface, remain unknown. For instance, does a neuron selective for 2D contour-defined shape prefer 3D solid surfaces or 2D flat surfaces? How are preferences for such heterogeneous attributes arranged in a common topographic map? To address these questions, we leverage V4 "digital twins" -- deep neural network models fitted to large-scale, wide-field calcium imaging data comprising tens of thousands of natural images. These digital twins allow us to systematically probe not only the stimulus dimensions explored in earlier studies, but also new, multidimensional stimulus sets that reveal additional aspects of the V4 code. In this study, we find that neural pixels preferring 2D contour-defined shapes also tend to prefer 3D surface shape defined by shading or texture gradients and by object form. In contrast, pixels preferring 2D texture tend to prefer flat surfaces defined by uniform texture or reflectance. We propose that this division of labor suggests that V4 may decompose the encoding of geometrical shape and surface appearance of visual stimuli into distinct populations of neurons, organized as interleaved clusters in the V4 topographic map.