The Neural Process Family: Survey, Applications and Perspectives

TL;DR

This paper provides a comprehensive survey of the Neural Processes Family, highlighting their ability to combine neural networks with probabilistic reasoning, and demonstrates their effectiveness across various data domains.

Contribution

It offers a detailed taxonomy, synthesizes recent research, and empirically evaluates Neural Processes Family models across multiple input dimensions.

Findings

Neural Processes effectively model data generating functions in 1D, 2D, and 3D domains.

The survey organizes motivation, methodology, and applications of NPF models.

Empirical results show NPF models outperform traditional methods in uncertainty estimation.

Abstract

The standard approaches to neural network implementation yield powerful function approximation capabilities but are limited in their abilities to learn meta representations and reason probabilistic uncertainties in their predictions. Gaussian processes, on the other hand, adopt the Bayesian learning scheme to estimate such uncertainties but are constrained by their efficiency and approximation capacity. The Neural Processes Family (NPF) intends to offer the best of both worlds by leveraging neural networks for meta-learning predictive uncertainties. Such potential has brought substantial research activity to the family in recent years. Therefore, a comprehensive survey of NPF models is needed to organize and relate their motivation, methodology, and experiments. This paper intends to address this gap while digging deeper into the formulation, research themes, and applications concerning the family members. We shed light on their potential to bring several recent advances in other deep learning domains under one umbrella. We then provide a rigorous taxonomy of the family and empirically demonstrate their capabilities for modeling data generating functions operating on 1-d, 2-d, and 3-d input domains. We conclude by discussing our perspectives on the promising directions that can fuel the research advances in the field. Code for our experiments will be made available at https://github.com/srvCodes/neural-processes-survey.