The macaque IT cortex but not current artificial vision networks encode object position in perceptually aligned coordinates

TL;DR

This study demonstrates that macaque IT cortex encodes object position in perceptually aligned coordinates, unlike current artificial vision networks, which lack this adaptive spatial coding, highlighting a key difference between biological and artificial visual processing.

Contribution

The paper reveals that macaque IT cortex encodes object position in perceptually meaningful coordinates and shows that artificial networks do not naturally replicate this adaptive spatial coding.

Findings

IT population codes shift with motion aftereffect, mirroring human perception

Artificial networks do not exhibit adaptation-induced position shifts

Transforming artificial features with IT adaptation dynamics reproduces biases

Abstract

Efficient interaction with the visual world requires not only accurate object identification but also precise localization of objects in space. While spatial ("where") processing has traditionally been attributed to dorsal stream pathways, recent work has shown that object position can also be decoded from responses in ventral stream areas such as the inferior temporal (IT) cortex. However, because object position in these paradigms is tightly coupled to pixel-based location, it remains unclear whether ventral stream position signals reflect perceptually meaningful spatial representations or simply inherited retinotopic structure. To address this question, we used the motion aftereffect, a classic visual illusion that shifts perceived object position without changing retinal input. Combining large-scale intracortical recordings in macaque IT with matched human psychophysics, we found that motion adaptation induces systematic direction-opponent biases in IT population codes for object position that mirror human perceptual reports, despite identical pixel-level stimuli. These effects are accompanied by adaptation-driven changes in the geometry of IT population representations. We further tested whether artificial vision systems exhibit similar dynamics. Standard feedforward, recurrent, and state-of-the-art video-based neural networks accurately encode object position but fail to produce adaptation-induced position shifts. However, applying empirically derived transformations based on IT adaptation dynamics to model feature spaces is sufficient to generate similar biases. Together, these results indicate that IT represents object position in perceptually aligned coordinates and also highlight a gap between biological and artificial vision systems in capturing history-dependent spatial coding.