Practical Conditional Neural Processes Via Tractable Dependent Predictions

TL;DR

This paper introduces a new class of neural process models that produce correlated predictions, are simple to train, scalable, and capable of modeling non-Gaussian outputs, improving performance in dependent estimation tasks.

Contribution

The authors develop a novel neural process framework that supports exact maximum likelihood training, models output dependencies efficiently, and incorporates invertible transformations for non-Gaussian distributions.

Findings

Models outperform existing approaches in dependent prediction tasks.

The approach is scalable and easy to train on large datasets.

Enhanced modeling of non-Gaussian output distributions.

Abstract

Conditional Neural Processes (CNPs; Garnelo et al., 2018a) are meta-learning models which leverage the flexibility of deep learning to produce well-calibrated predictions and naturally handle off-the-grid and missing data. CNPs scale to large datasets and train with ease. Due to these features, CNPs appear well-suited to tasks from environmental sciences or healthcare. Unfortunately, CNPs do not produce correlated predictions, making them fundamentally inappropriate for many estimation and decision making tasks. Predicting heat waves or floods, for example, requires modelling dependencies in temperature or precipitation over time and space. Existing approaches which model output dependencies, such as Neural Processes (NPs; Garnelo et al., 2018b) or the FullConvGNP (Bruinsma et al., 2021), are either complicated to train or prohibitively expensive. What is needed is an approach which provides dependent predictions, but is simple to train and computationally tractable. In this work, we present a new class of Neural Process models that make correlated predictions and support exact maximum likelihood training that is simple and scalable. We extend the proposed models by using invertible output transformations, to capture non-Gaussian output distributions. Our models can be used in downstream estimation tasks which require dependent function samples. By accounting for output dependencies, our models show improved predictive performance on a range of experiments with synthetic and real data.