Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity

TL;DR

This paper introduces TE-ViDS, a sequential latent variable model that captures the dynamic neural activity in the mouse visual cortex, improving decoding and interpretability of neural responses to naturalistic stimuli.

Contribution

The study presents TE-ViDS, a novel time-evolving dynamical system that explicitly models temporal neural dynamics and uses contrastive learning to enhance latent representations.

Findings

TE-ViDS achieves superior decoding performance on neural data.

It uncovers interpretable neural trajectories.

It reveals differences in visual processing across subjects and regions.

Abstract

Seeking high-quality representations with latent variable models (LVMs) to reveal the intrinsic correlation between neural activity and behavior or sensory stimuli has attracted much interest. In the study of the biological visual system, naturalistic visual stimuli are inherently high-dimensional and time-dependent, leading to intricate dynamics within visual neural activity. However, most work on LVMs has not explicitly considered neural temporal relationships. To cope with such conditions, we propose Time-Evolving Visual Dynamical System (TE-ViDS), a sequential LVM that decomposes neural activity into low-dimensional latent representations that evolve over time. To better align the model with the characteristics of visual neural activity, we split latent representations into two parts and apply contrastive learning to shape them. Extensive experiments on synthetic datasets and real neural datasets from the mouse visual cortex demonstrate that TE-ViDS achieves the best decoding performance on naturalistic scenes/movies, extracts interpretable latent trajectories that uncover clear underlying neural dynamics, and provides new insights into differences in visual information processing between subjects and between cortical regions. In summary, TE-ViDS is markedly competent in extracting stimulus-relevant embeddings from visual neural activity and contributes to the understanding of visual processing mechanisms. Our codes are available at https://github.com/Grasshlw/Time-Evolving-Visual-Dynamical-System.