Interpreting V1 Population Activity via Image-Neural Latent Representation Alignment

TL;DR

DINA is an interpretable framework that aligns visual stimuli with V1 neural responses in a shared latent space, enabling accurate decoding and insights into visual processing mechanisms.

Contribution

The paper introduces DINA, a dual-tower architecture that jointly aligns visual stimuli and neural responses, providing both decoding accuracy and interpretability of neural computations.

Findings

Decoding performance relies on coarse visual structures rather than semantic details.

Alignable feature maps emerge from multiple spatial regions, capturing shape and texture.

Sparse subsets of neurons primarily reconstruct these feature maps.

Abstract

Understanding the neural mechanisms underlying visual computation has long been a central challenge in neuroscience. Recent alignment based approaches have improved the accuracy of decoding visual stimuli from brain activity, yet they provide limited insight into the neural computations that give rise to these improvements. To address this gap, we propose Dual-Tower Image-Neural Alignment (DINA), an interpretable contrastive framework for analyzing population level visual computations in primary visual cortex (V1). DINA jointly trains a biologically motivated dual-tower architecture that aligns visual stimuli and corresponding V1 population responses in a shared latent space at the level of intermediate feature maps, enabling both accurate decoding and direct access to interpretable feature maps. Evaluated on large-scale two-photon calcium imaging data from mouse V1, DINA achieves accurate neural-based decoding while revealing that decoding performance is primarily supported by coarse, low-level visual structure, rather than semantic category information or fine-grained details. Further analysis reveals that alignable feature maps emerge from multiple spatially distributed image regions, capturing both shape and texture cues, and are predominantly reconstructed by sparse subsets of strongly responsive neurons and their functional interactions. Together, these results confirm that, beyond enabling accurate decoding, DINA provides a principled framework for probing the computational mechanisms underlying visual processing in V1.