Factorized Neural Processes for Neural Processes: $K$-Shot Prediction of Neural Responses

TL;DR

This paper introduces a Factorized Neural Process model for rapid $K$-shot prediction of neural responses, enabling real-time neural tuning inference from minimal data, outperforming traditional optimization methods in speed and accuracy.

Contribution

The paper develops a novel Factorized Neural Process that efficiently infers neural tuning functions from few examples, significantly reducing inference time compared to existing methods.

Findings

Accurate prediction of neural responses with few stimulus-response pairs.

Fast inference through a single forward pass of the neural network.

Comparable or superior predictive accuracy to optimization-based approaches.

Abstract

In recent years, artificial neural networks have achieved state-of-the-art performance for predicting the responses of neurons in the visual cortex to natural stimuli. However, they require a time consuming parameter optimization process for accurately modeling the tuning function of newly observed neurons, which prohibits many applications including real-time, closed-loop experiments. We overcome this limitation by formulating the problem as $K$-shot prediction to directly infer a neuron's tuning function from a small set of stimulus-response pairs using a Neural Process. This required us to developed a Factorized Neural Process, which embeds the observed set into a latent space partitioned into the receptive field location and the tuning function properties. We show on simulated responses that the predictions and reconstructed receptive fields from the Factorized Neural Process approach ground truth with increasing number of trials. Critically, the latent representation that summarizes the tuning function of a neuron is inferred in a quick, single forward pass through the network. Finally, we validate this approach on real neural data from visual cortex and find that the predictive accuracy is comparable to -- and for small $K$ even greater than -- optimization based approaches, while being substantially faster. We believe this novel deep learning systems identification framework will facilitate better real-time integration of artificial neural network modeling into neuroscience experiments.