Unsupervised learning of multiscale switching dynamical system models from multimodal neural data

TL;DR

This paper introduces an unsupervised learning method for multiscale switching dynamical systems that effectively models multimodal neural data, capturing regime-dependent non-stationarity to improve behavior decoding.

Contribution

The paper presents a novel unsupervised algorithm for learning multiscale switching dynamical models from multimodal neural recordings, addressing the challenge of regime labels and leveraging multiple data modalities.

Findings

Multiscale models outperform single-scale models in decoding behavior.

Models incorporating regime switches outperform stationary models.

Multimodal fusion enhances neural dynamics modeling.

Abstract

Neural population activity often exhibits regime-dependent non-stationarity in the form of switching dynamics. Learning accurate switching dynamical system models can reveal how behavior is encoded in neural activity. Existing switching approaches have primarily focused on learning models from a single neural modality, either continuous Gaussian signals or discrete Poisson signals. However, multiple neural modalities are often recorded simultaneously to measure different spatiotemporal scales of brain activity, and all these modalities can encode behavior. Moreover, regime labels are typically unavailable in training data, posing a significant challenge for learning models of regime-dependent switching dynamics. To address these challenges, we develop a novel unsupervised learning algorithm that learns the parameters of switching multiscale dynamical system models using only multiscale neural observations. We demonstrate our method using both simulations and two distinct experimental datasets with multimodal spike-LFP observations during different motor tasks. We find that our switching multiscale dynamical system models more accurately decode behavior than switching single-scale dynamical models, showing the success of multiscale neural fusion. Further, our models outperform stationary multiscale models, illustrating the importance of tracking regime-dependent non-stationarity in multimodal neural data. The developed unsupervised learning framework enables more accurate modeling of complex multiscale neural dynamics by leveraging information in multimodal recordings while incorporating regime switches. This approach holds promise for improving the performance and robustness of brain-computer interfaces over time and for advancing our understanding of the neural basis of behavior.