Neural Processes

TL;DR

Neural Processes are a new class of models that combine the data efficiency and uncertainty estimation of Gaussian processes with the computational efficiency of neural networks, enabling rapid adaptation and flexible function modeling.

Contribution

Introduction of Neural Processes, a novel neural latent variable model that merges the strengths of Gaussian processes and neural networks for flexible, efficient, and probabilistic function modeling.

Findings

Neural Processes perform well on regression and optimization tasks.

NPs can rapidly adapt to new data with uncertainty estimates.

They are computationally efficient during training and inference.

Abstract

A neural network (NN) is a parameterised function that can be tuned via gradient descent to approximate a labelled collection of data with high precision. A Gaussian process (GP), on the other hand, is a probabilistic model that defines a distribution over possible functions, and is updated in light of data via the rules of probabilistic inference. GPs are probabilistic, data-efficient and flexible, however they are also computationally intensive and thus limited in their applicability. We introduce a class of neural latent variable models which we call Neural Processes (NPs), combining the best of both worlds. Like GPs, NPs define distributions over functions, are capable of rapid adaptation to new observations, and can estimate the uncertainty in their predictions. Like NNs, NPs are computationally efficient during training and evaluation but also learn to adapt their priors to data. We demonstrate the performance of NPs on a range of learning tasks, including regression and optimisation, and compare and contrast with related models in the literature.