Deep inference of latent dynamics with spatio-temporal super-resolution using selective backpropagation through time

TL;DR

This paper introduces a novel neural network training method called SBTT that enables spatio-temporal super-resolution in neural data, improving the inference of latent neural dynamics from bandwidth-limited recordings.

Contribution

The paper presents SBTT, a new training strategy for deep generative models that infers missing neural activity and enhances resolution in electrophysiological and calcium imaging data.

Findings

SBTT enables accurate inference of neural population dynamics at lower bandwidths.

The method outperforms current state-of-the-art in calcium imaging data analysis.

Pretraining with high-bandwidth data improves performance on sparse samples.

Abstract

Modern neural interfaces allow access to the activity of up to a million neurons within brain circuits. However, bandwidth limits often create a trade-off between greater spatial sampling (more channels or pixels) and the temporal frequency of sampling. Here we demonstrate that it is possible to obtain spatio-temporal super-resolution in neuronal time series by exploiting relationships among neurons, embedded in latent low-dimensional population dynamics. Our novel neural network training strategy, selective backpropagation through time (SBTT), enables learning of deep generative models of latent dynamics from data in which the set of observed variables changes at each time step. The resulting models are able to infer activity for missing samples by combining observations with learned latent dynamics. We test SBTT applied to sequential autoencoders and demonstrate more efficient and higher-fidelity characterization of neural population dynamics in electrophysiological and calcium imaging data. In electrophysiology, SBTT enables accurate inference of neuronal population dynamics with lower interface bandwidths, providing an avenue to significant power savings for implanted neuroelectronic interfaces. In applications to two-photon calcium imaging, SBTT accurately uncovers high-frequency temporal structure underlying neural population activity, substantially outperforming the current state-of-the-art. Finally, we demonstrate that performance could be further improved by using limited, high-bandwidth sampling to pretrain dynamics models, and then using SBTT to adapt these models for sparsely-sampled data.