Multi-Modal Latent Variables for Cross-Individual Primary Visual Cortex Modeling and Analysis

TL;DR

This paper introduces a multi-modal variational autoencoder that effectively models and analyzes neural activity and visual stimuli in the primary visual cortex across different individuals, improving cross-individual alignment and interpretability.

Contribution

The study presents a novel multi-modal identifiable variational autoencoder with a two-level disentanglement strategy for cross-individual neural data analysis, enabling robust latent space mapping and interpretation.

Findings

Achieves state-of-the-art cross-individual latent representation and alignment.

Identifies neuronal subpopulations with distinct temporal and stimulus features.

Reveals stimulus sensitivities to edge features and luminance variations.

Abstract

Elucidating the functional mechanisms of the primary visual cortex (V1) remains a fundamental challenge in systems neuroscience. Current computational models face two critical limitations, namely the challenge of cross-modal integration between partial neural recordings and complex visual stimuli, and the inherent variability in neural characteristics across individuals, including differences in neuron populations and firing patterns. To address these challenges, we present a multi-modal identifiable variational autoencoder (miVAE) that employs a two-level disentanglement strategy to map neural activity and visual stimuli into a unified latent space. This framework enables robust identification of cross-modal correlations through refined latent space modeling. We complement this with a novel score-based attribution analysis that traces latent variables back to their origins in the source data space. Evaluation on a large-scale mouse V1 dataset demonstrates that our method achieves state-of-the-art performance in cross-individual latent representation and alignment, without requiring subject-specific fine-tuning, and exhibits improved performance with increasing data size. Significantly, our attribution algorithm successfully identifies distinct neuronal subpopulations characterized by unique temporal patterns and stimulus discrimination properties, while simultaneously revealing stimulus regions that show specific sensitivity to edge features and luminance variations. This scalable framework offers promising applications not only for advancing V1 research but also for broader investigations in neuroscience.