Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors

TL;DR

This paper introduces a probabilistic dynamic encoding model that combines naturalistic video stimuli with stimulus-independent latent factors to better predict neural responses in mouse visual cortex, revealing meaningful biological structures.

Contribution

It presents the first dynamic neural encoding model explicitly incorporating latent states and the entire response distribution, trained on mouse V1 data.

Findings

Outperforms video-only models in likelihood and correlation.

Latent factors correlate with mouse behavior.

Latent patterns relate to cortical neuron positions.

Abstract

The neural activity in the visual processing is influenced by both external stimuli and internal brain states. Ideally, a neural predictive model should account for both of them. Currently, there are no dynamic encoding models that explicitly model a latent state and the entire neuronal response distribution. We address this gap by proposing a probabilistic model that predicts the joint distribution of the neuronal responses from video stimuli and stimulus-independent latent factors. After training and testing our model on mouse V1 neuronal responses, we find that it outperforms video-only models in terms of log-likelihood and achieves improvements in likelihood and correlation when conditioned on responses from other neurons. Furthermore, we find that the learned latent factors strongly correlate with mouse behavior and that they exhibit patterns related to the neurons' position on the visual cortex, although the model was trained without behavior and cortical coordinates. Our findings demonstrate that unsupervised learning of latent factors from population responses can reveal biologically meaningful structure that bridges sensory processing and behavior, without requiring explicit behavioral annotations during training.