In silico discovery of representational relationships across visual cortex

TL;DR

This study introduces Relational Neural Control (RNC), a novel method to explore and validate the representational relationships across visual cortex areas using in silico and in vivo fMRI data, revealing network-level organization.

Contribution

The paper develops RNC to investigate cortical representational relationships and validates in silico findings with empirical fMRI data, advancing understanding of visual cortex organization.

Findings

Shared and unique representations vary with cortical distance and hierarchy.

In silico responses can control and disentangle responses across visual areas.

Empirical validation confirms in silico predictions about cortical relationships.

Abstract

Now published in Nature Human Behavior doi: https://doi.org/10.1038/s41562-025-02252-z Human vision is mediated by a complex interconnected network of cortical brain areas that jointly represent visual information. While these areas are increasingly understood in isolation, their representational relationships remain elusive. Here we developed relational neural control (RNC), and used it to investigate the representational relationships for univariate and multivariate fMRI responses of areas across visual cortex. Through RNC we generated and explored in silico fMRI responses for large amounts of images, discovering controlling images that align or disentangle responses across areas, thus indicating their shared or unique representational content. This revealed a typical network-level configuration of representational relationships in which shared or unique representational content varied based on cortical distance, categorical selectivity, and position within the visual hierarchy. Closing the empirical cycle, we validated the in silico discoveries on in vivo fMRI responses from independent subjects. Together, this reveals how visual areas jointly represent the world as an interconnected network.